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ABSTRACT 

The  control  of  linear  systems  with  incomplete  information  is  considered 
where  the  unknown  disturbances  and/or  random  parameters  are  assumed  to 
satisfy  seme  statistical  laws. 

The  observer  theory  for  linear  systems  is  developed  which  generalizes 
the  concepts  due  to  Kalman  and  Luenberger  pertaining  to  the  design  of  linear 
systems  which  estimate  the  state  f  a  linear  plant  on  the  basis  of  both 
noise-free  and  noisy  measurements  of  the  output  variables.  The  Separation 
Theorem  for  linear  system  is  then  extended  for  such  observers-estimators. 

The  problem  of  controlling  a  linear  system  with  unknown  gain  is  then 
considered.  An  open-loop-feedback-optimal  control  algorithm  is  developed 
which  seems  to  be  computationally  feasible.  Existence  of  such  suboptimal 
control  scheme  is  proved  under  the  assumption  that  the  uncertainties  in  the 
unknown  gain  will  not  grow  in  time.  Convergence  of  such  suboptimal  control 
system  to  the  truly  optimal  control  system  is  considered.  A  computer  pro¬ 
gram  is  developed  to  study  the  control  of  a  variety  thrid  order  systems 
with  known  poles  but  unknown  zeroes.  The  experimental  results  serve  to  pro¬ 
vide  us  with  some  more  insights  into  the  structure  and  behavior  of  the 
open-loop-feedback-optimal  control  systems. 
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CHAPTER  I 


INTRODUCTION 

In  recenc  years,  deterministic  optimal  control  theory  has  come  to  its 
full  maturity.  Text  books  [57],  [43]  have  been  written  which  are  devoted 
to  the  theory  and  application  of  modem  control  theory.  In  deterministic 
control  theory,  it  is  assumed  that  the  future  effect  of  any  present  control 
action  is  exactly  known;  this  class  of  control  problems  is  often  called  con¬ 
trol  with  complete  information  [73].  In  many  situations,  the  necessity  of 
control  arises  from  the  fact  that  there  are  perturbing  disturbances  and/or 
component  failures  in  the  physical  system.  These  uncertain  phenomena  pre¬ 
vent  us  to  deduce  exactly  the  future  effect  of  all  present  actions,  and 
thus  deterministic  control  theory  may  not  be  strictly  applicable.  The 
classes  of  control  proolems  where  future  effect  cannot  be  predicted  exactly 
are  called  control  with  incomplete  information.  There  are  cases  where  the 
uncertain  phenomena  can  be  appropriately  modelled  as  stochastic  processes, 
so  stochastic  models  and  stochastic  control  theory  can  be  applied  [4],  [74]. 
There  are  also  cases  where  the  chance  phenomena  have  no  statistical  regu¬ 
larity,  in  these  situations,  the  game-theoretic  approach  [65]  to  obtain 
min-max  control  may  be  more  appropriate. 

In  this  thesis  we  shall  study  some  classes  of  problems  with  incomplete 
information.  First  we  assume  that  the  system  beinr  controlled  is  linear 
(either  discrete  time  or  continuous  time).  The  disturbance  and  random 
parameters  are  assumed  to  satisfy  some  statistical  laws.  In  the  beginning, 
we  assume  that  the  only  sources  of  uncertainty  are  the  driving  and/or  obser¬ 
vation  disturbances.  The  statistical  laws  of  disturbances  are  assumed  to  be 
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imea,  we  shall  consider  the  case  where  some  parameters  of  the  system 


arc  cnkraswa  bur  satisfy  some  statistical  laws. 

In  Chapter  II,  same  mathematical  preliminaries  are  collected  for-  ease 
of  references.  Probability  theory  is  created  briefly  from  a  measure- 
theoretic  approach.  Facts  about  linear  stochastic  difference  (and  differesa- 
tial)  equations,  and  stochastic  optimization  problems  are  included  for  the 
sake  of  camp lateness.  The  sections  on  Generalized  Riccati  Equations  are 
new  results  and  will  be  useful  in  later  discussions.  The  theory  for  ob¬ 
servers  for  discrete  time  ana  continuous  tine  linear  systems  is  developed 
in  Chapters  III  and  IV.  The  conceptual  framework  is  that  an  observer  Is  a 
device  which  will  supply  complementary  information  about  all  recoverable 
uncertainties.  The  observer  theory  is  applied  to  estimation  problems  where: 
we  have  only  partial  observation  of  the  states  in  the  presence  of  observa¬ 
tion  ncine  which  may  be  degenerate  or  even  totally  singular.  The  results 
will  include  the  Kalman  filter  139],  [50]  as  a  special  case¬ 
in  Chapter  V,  we  consider  the  optimal  control  of  linear  systea  with 
known  dynamics  with  respect  to  quadratic  criterion.  The  uncertainties 
arise  from  driving  and/or  observation  disturbances  with  known  statistical 
laws.  One  such  class  of  problems  had  been  considered  before  by  Joseph  and 
Tou  [56],  Streibel  [59J  and  Vonhan  [22].  They  made  the  assumption  chat  the: 
observation  noise  is  nondegenerate  Gaussian  white  noise  process  (see  Section 
2.2).  In  this  work,  this  assumption  is  relaxed.  It  is  assumed  that  the 
observation  noise  may  be:  1)  nondegenerate  Gaussian  white  noise,  2)  de¬ 
generate  Gaussian  white  noise,  3)  colored  observation  noise,  4)  totally 
singular  (i.e.,  noise-free  observations)  or  5)  the  sun  of  colored  and  white 


Gaussian  noise.  The  approach  follows  that  of  Wonham's  [22]  and  the  tech¬ 
nique  is  the  dynamic  programming  method. 

The  control  of  linear  systems  with  unknown  gain  parameters  is  con- 
s.dered  in  Chapter  VI.  The  open-loop-feedback-optiraal  approach  is  used  to 
derive  a  suboptimal  control  sequence  which  appears  to  be  computationally 
feasible.  The  technique  used  is  that  of  the  matrix  minimum  principle.  Ana¬ 
lytical  studies  on  the  overall  suboptimal  control  system  are  carried  out 
and  the  asymptotic  behavior  of  the  overall  suboptimal  control  system  is  de¬ 
rived.  Computer  simulations  for  some  third  order  linear  systems  were  carried 
out  based  on  the  theoretical  results  obtained  in  Chapter  VI.  The  experi¬ 
mental  results  are  discussed  in  Chapter  VII.  Conclusions  and  some  topics 
for  further  research  are  listed  in  Chapter  VIII. 

The  perspective  and  comparison  of  this  work  with  published  references 
are  done  at  the  end  of  each  chapter.  In  this  contribution,  we  develop  the 
observer  theory  which  provides  a  deeper  understanding  of  the  structure  of 
state  estimators  in  the  case  of  nondegenerate,  degenerate,  singular,  or 
colored  observation  noise.  Thi  theory  unifies  some  seemingly  different 
concepts  of  Kalman  filter,  Luenberger  observer  and  exponential  estimator, 
and  treated  them  in  one  general  framework.  Then  we  have  the  extension  of 
the  Separation  Theorem  for  such  observers-astimators .  Finally,  we  have  de¬ 
veloped  the  open-loop  feedback  optimal  control  algorithm  for  the  linear 
stochastic  systems  with  unknown  constant  or  random  gain  parameters;  theo¬ 
retical  and  experimental  studies  are  carried  out  to  this  class  of  problems 
which  provide  us  with  some  insights  into  the  structure  and  behavior  of  the 


overall  control  system. 


Notations : 


Lower  case  underscored  letters  stand  for  vectors  (e.g.,  jc,  y) ;  upper 
case  underscored  letters  stand  for  matrices  (e.g..  A,  B) .  Noise  distur¬ 
bances  are  denoted  by  lower  case  underscored  Greek  letters  (e.g.,  C.,  n) . 
Lower  case  letters  with  subscripts  will  denote  components  (e.g.,  x^  will  be 
the  i-th  component  of  the  vector  x.»  will  be  the  ij-th  element  of  matrix 
A). 


The  transpose  of  a  matrix  A  is  denoted  by  A’ .  The  transpose  of  a 
column  vector,  x,  is  a  row  vector  and  is  denoted  by  x' . 

Let  A  be  an  nxn  square  matrix;  the  trace  of  A  is  defined  as 


Er  A  ■  I  aii  ' 

i=l 

Let  H(x.  ,  x  ,  x  )  be  a  scalar  function;  we  shall  denote  it  by 

J.  JL  X/  T1ITI 


H(X)  •  The  gradient  matrix  is  defined  by 

r  3H(  y  •  •  • 


f =(- 


3H(X) 
3X 


,  x  ) 
nm 


3x.  . 
ij 


M  will  denote  the  set  of  all  nXm  .-atrices. 
nm 

R^  will  denote  the  product  space  of  ordered  n-tuples  of  real  numbers, 

we  shall  denote  the  elements  in  R  by  column  vector  x. 

n  — 

I  will  denote  the  set  of  all  integers  and  Ir.  . .  will  denote  the  set  of 

[i,j] 


integers  {i,  i  +  1,  ...,  j},  i  £  j. 


CHAPTER  II 


MATHEMATICAL  PRELIMINARIES 


2.1  Introduction 

The  purpose  of  this  chapter  is  to  introduce  the  mathematical  results 
which  will  be  used  frequently  in  the  later  chapters.  Some  of  these  results 
are  known  in  the  literature  while  soma  are  due  mainly  to  the  author. 

In  Section  2.2,  probability  theory  is  treated  briefly  using  the 
measure-theoretic  approach.  Except  for  the  precise  basic  definitions,  the 
treatment  is  physical  rather  than  mathematical.  For  a  detailed  and  rigorous 
mathematical  treatment,  see  Doob  [1]  and  Loeve  [2].  A  rigorous  mathematical 
consideration  on  conditional  expectation  and  conditional  distribution  of  a 
random  vector  is  given.  In  the  opinion  of  the  author,  a  thorough  under¬ 
standing  of  these  concepts  is  vital  in  most  stochastic  optimization  problems. 

Jn  Section  2.3,  linear  stochastic  difference  and  differential  equations 
are  treated  to  the  extent  that  some  of  the  discussions  in  later  chapters 
will  require  for  the  sake  of  completeness. 

In  Section  2.4,  the  matrix  minimum  principle  and  optimality  criteria 
are  considered  to  some  detail.  The  matrix  minimum  principle  can  allow  us 
to  deduce  the  necessary  conditions  for  optimality  for  some  special  problems, 
whereas  the  optimality  criteria  provides  us  a  test  to  see  whether  a  certain 
solution  is  optimal. 

In  most  control  and  filtering  problems,  we  shall  encounter  a  matrix 
Riccati  Difference  or  Differential  equation.  To  foresee  thare  generalized 
matrix  Riccati  difference  and  differential  equations  are  investigated  in 
detail  in  sections  2.5  and  2.6.  The  results  obtained  in  these  sections 
are  new,  while  the  approach  follows  that  of  Wonham's  [32] . 
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From  Section  2.2  to  Section  2.4,  the  results  are  Uncvn.  The  discus¬ 


sions  in  these  sections  are  by  *io  means  exhaustive:  detailed  references 
are  given  in  Section  2.2  to  indicate  where  core  extensive  results  can  be 
found. 


2.2  Probability  Theory 

Definition  2.2.1:  Let  u  be  a  set.  A  3-aigebra  (iJorei  Field,  o-field)  on 
H,  F,  is  a  class  of  subsets  of  2,  such  that  it  has  the  following  properties: 


a) 

fi  e  F 

*o) 

If  A  c  F, 

then  A* 

c) 

If  A,  £  F, 

II 

■H 

U  A.  c  F 
5=1  * 


n  A.  t  F 

l 

i=l 


The  pair  (2,  F)  consisting  of  a  set  fi  and  a  c-fieAd  F  of  subsets  of  SI 
is  called  a  measurable  space.  The  elements  of  2  are  called  F-neasurable 
sets,  or  just  measurable  sets  if  there  is  no  ambiguity.  In  probability 
theory,  the  set  2  represents  the  sample  space,  and  F  represents  the  collec¬ 
tions  of  possible  events. 

Definition  2.2.2:  Let  (SI  ,  F,)  and  (Si  ,  F_)  be  two  measurable  spaces.  A 
napping  f  of  2^  onto  2^  is  said  to  be  measurable  if  it  sati-.fies  the  condition 


f  X(A)  z  for  every  A  c 


Definition  2.2.3:  Let  2  be  a  set,  and  a  family  or  mappings  of  ‘1 

into  measurable  spaces  (2^,  The  a-algebra  generated  by  is 

the  smallest  o-algebra  on  2  with  respect  to  which  all  functions  (f^) .  ^  are 
measurable,  and  is  denoted  by  F(f^,  i  e  I). 

T  C 

A  denotes  the  complement  of  A. 


-7- 


Frcm  the  above  two  definitions,  we  see  that  if  F'  is  the  o-aJgebra 
generated  by  <f f) i£; ^  t  and  F"  is  the  o-algebra  generated  by  (r ic. -j;,,  Chen 
F«  c  F"  if  and  only  if  I*  *=  I"  while  I'  and  1"  are  both  countable. 

In  general,  a  basic  measurable  space  (ft,  F)  is  assumed  to  be  given 
which  describes  the  underlying  uncertainty  of  random  phenomena.  Such  a 
measurable  space  is  of  an  abstract  nature;  how  the  uncertainties  reveal 
themselves  depends  on  the  type  of  experiments  we  perform  to  obtain  observa¬ 
tions,  the  outcomes  of  which  we  usually  referred  to  as  statistics.  In  ab¬ 
stract  mathematical  formulation,  we  let  (ft^,  F^)  be  another  measurable 
space,  where  we  call  ft^  the  observation  space  and  the  collections  of  all 
possible  observations.  A  measurable  function,  f,  from  ft  to  ft^  is  callrd  the 
observation  statistic.  Let  F  C  F  be  a  sub-o-algebra,  an  observation 
statistic,  f,  is  said  to  be  F-measurable  if  F(f)  cF.  Special  cases  of 
observation  statistics  are  random  vectors  (ft^  =  RR)  and  random  processes 
is  the  set  of  functions  defined  on  [o,  T]  with  values  in  Rn) . 

Definition  2.2.4:  Let  (ft,  F)  be  a  measurable  space.  A  probability  law 
on  this  space  is  an  abstract  positive  measure  u  defined  on  F, '  and  having 
u(ft)  =  1.  The  triplet  (ft,  F,  p)  is  called  a  probability  space. 

Let  (ft,  F,  u)  be  a  basic  probability  space,  and  let  (ft^,  F^)  be  another 
measurable  space  representing  the  observations  with  a  statistic  f  which  maps 
ft  onto  ft^.  We  can  define  a  probability  law  on  (ft  ,  F^)  by  defining 
Uf(A)  =  p(f  ^(A));  A  t  F^ .  We  shall  call  p^  the  statistical  law  of  u  under 
f;  this  law  is  also  called  the  law  of  distribution  of  the  statistic  f. 

p(*)  is  a  set  function  defined  on  F  with  the  property  of  countable  addi¬ 
tivity,  i.e.,  if  A^  c  F,  n  e  I,  are  disjoint,  then  we  have 

“(u  *„)- E  u{\)  ■ 

n  n 
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Definition  2.2.5:  Let  (ft,  F,  p)  be  a  probability  space,  let  be  a  sub- 
cr-field  of  Fr  and  let  x  be  an  integrable  real-valued  random  vector.  A 
conditional  expectation  of  x  relative  to  F^  is  an  integrable  F^-measurable 
real-valued  random  vector  such  that 

[  x(w)dp(w)  =  I  ,y.(w)dp(w)  for  every  A  e  F.  .  (2.2.1) 

U  SA  1 

By  the  Radon-Nikodym  theorem,  such  a  random  vector,  exists  and  is  unique 
a.s.  (almost  surely):  i.e.,  if  y_'  is  another  random  vector  satisfying 
(2.2.1),  then 


u(w:  ^(w)  =  (u)}  =  1  .  (2.2.2) 

For  this  reason,  we  may  simply  write  such  as  E{x|F-}.  The  conditional  ex¬ 
pectation  of  the  indicator  of  A  e  F  with  respect  to  F^,  Et^jF^},  is  also 
called  the  conditional  probability  of  A  relative  to  F^.  Note  that  this 
"probability"  is  a  random  variable  defined  up  to  an  a.s.  equality,  and  not 
a  number. 

Lemma  2.2.6:  Let  (ft,  F,  p)  be  a  probability  space.  Let  F^,  F2  be  sub-o- 
algebra  of  F  with  F^  C  F2>  Then 

E|E{yjF2}|  Fj  =  EjvjFj  a.s.  (2.2.3) 

where  is  any  p-integrable  real-valued  random  vector. 

Proof :  By  definition  2.2.5,  we  have  for  all  A  e 

j  E{2|F2}dy  =  j  X  dp  .  (2.2.4) 

A  A 

By  assumption,  C  therefore  (2.2.4)  holds  for  all  A  e  F^.  Therefore 


(2.2.5) 
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f  EiEklF2flFifdli  =  f  E{^dv  - 

J  A  1 A 

j  x  dn  -  |  E{Z|F1}dp  ;  A  e  Fx 
A  A 

Mow  (2.2.4)  follows  from  the  a.s.  uniqueness  of  (2.2.1). 

Lemma  2.2.7:  Let  (ft,  F,  y)  be  a  probability  space.  Let  F^  be  a  sub-o- 
algebra  of  F.  Let  y  be  a  u-integrable  random  variable  and  x  is  a  F  - 
measurable  random  variable,  then 

E{xy|Fx}  =  x  Ejy|F1}  .  (2.2.6) 

Equation  (2.2.6)  is  true  when  x  is  a  simple  function,  and  the  general  case 
follows  using  the  approximation  procedure.  For  a  detailed  discussion,  see 
[1],  [2]. 

Let  f  be  an  observation  statistic  on  ft;  i.e.,  f  is  c  measurable  func¬ 
tion  from  (ft,  F)  onto  (ft^,  F^).  Let  F(f)  be  the  o-algebra  generated  by  f. 

Such  a  statistic  induces  a  conditional  probability  E{IA|F(0|on  F.  If  there 
exists  a  function  P^(A,  w)  such  that  for  each  us  t  ft,  P^(A,  us)  defines  a 
probability  measure  on  F  and  for  fixed  A  e  F. 

Pf (A,  us)  =  E)lA|F(f)[  a.s.  (2.2.7) 

then  P^(A,  u>)  is  called  a  conditional  measure  on  F  relative  to  the  statistics 
f.  Unfortunately,  such  P^(A,  u>)  may  not  exist,  and  so  it  may  not  always  be 
possible  to  define  a  conditional  measure  on  F  relative  to  a  certain  sta¬ 
tistic.  ^  Let  g  be  another  statistic  and  F(g)  is  the  o-algebra  generated 
by  g.  If  there  exists  a  conditional  measure  defined  on  F(g);  i.e.,  if  there 
is  a  function  P^(A,  w)  such  that  for  each  uueD,Pf:(A,U))  defines  a  measure  on  F(g), 
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for  fixed  A  e  F(g) 

Pf(A.  w)  =  E{l  jF(f)}  a.s.  (2.2.8) 

then  one  can  define  the  law  of  distribution  of  g  in  the  regular  manner. 

Doob^  had  proved  that  if  the  statistic  g  is  a  random  vector  (say  y  £  Rn) 
then  the  conditional  measure  on  F(y),  P^(A,  m) ,  A  e  F(y),  exists  and  so  the 
conditional  distribution  of  y  is  well  defined  (a.s.).  Let  us  denote  the 
conditional  distribution  by  P ^(Y,  to)  which  defines  a  conditional  measure 
on  the  Borel  set  of  Rn  through  y.  If  <{>(y)  is  a  measurable  function  of  n- 
variables  w-th  values  in  Rra,  then  almost  surely,  we  have:^ 

rco  #00 

E|*(y)lF(f)}  =  I  ...  *(y)  pf(dy  ...  dy  ,  to)  .  (2.2.9) 

We  can  visualize  F(f)  as  the  o-algebra  which  contains,  in  a  loose  sense, 
all  the  statistical  information  conveyed  by  the  observation  statistic  about 
the  total  underlying  uncertainty  of  the  basic  sample  space.  On  the  other  hand, 
the  conditional  measure  P^(A,  w) ,  A  c  F(y),  describes  the  statistical  infor¬ 
mation  of  f  conveyed  about  the  random  vector  y.  In  view  of  this  intuitive 
interpretation  we  have  the  following  definition. 

Definition  2.2.8:  Let  (ft,  F,  n)  be  a  probability  space,  and  let  F^,  F^  and 
F^  be  sub-o-algebras  of  F.  F^  and  F^  are  said  to  be  conditionally  independent 
relative  to  F2  if  for  any  random  vectors,  y^  which  is  F^-measurable,  and  y^ 
which  is  F^-measurable;  we  have 

1'3 |F2}  =  E{y1|F2(  E{y'|F2|  .  (2.2.10) 

Let  f^,  f2  be  two  observation  statistics;  f^  and  f2  are  said  to  be 
independent  if  F(f^)  and  F(f2)  are  conditional  independent  relative  to 


^2  =  3),  or  we  say  F(f^)  and  F(f2)  are  independent. 
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Let  (ft,  F,  p)  be  a  basic  probability  space,  and  let  £  be  a  random  n- 
vector  on  ft;  this  induces  a  law  of  distribution  on  Rn  through  the  statistic 
Let  F^CF  such  that  F^  and  F(y)  is  independent  relative  to  =  {$,  ft). 
Then  for  arbitrary  B  e  F^,  we  have  tor  A  e  F(;£),  and  measurable  in  R°: 

(  E|^(x)  |  F  }  dp  =  [  ±(%)  dp  =  j  I  ±(%)  dp 
'  B  JB  Jft 


=  dp  [  ±(x)  dp 

B  Jft 


( j 


dp  B  e  F. 


(2.2.11) 


B  Jft 


Therefore,  we  have 


e{<K2)|f.  }  =  f  4>(x)  dp 
x  Jo 


(2.2.12) 


In  particular  if  j>(%)  =•  I  ,  A  e  F(%),  then  (2.2.11)  and  (2.2.12)  become 


p(A  H  B)  =  p (A)  p (B) 


EjljF^  =  p  (A)  a.s.  .  (2.2.13) 


This  implies  that  if  F.  and  F(y)  are  independent,  the  conditional  distribu¬ 
tion  of  x  relative  to  F,  is  the  same  as  the  unconditional)  distribution  of  y_. 
Physically,  this  says  that  F^  reveals  no  information  about  In  many  cases, 
F^  is  generated  by  some  observation  statistics,  f^,  ...,  f^;  so  if  £  is  in¬ 
dependent  of  F^  =  F(f^,  i  =  1,  . . . ,  n) ,  this  means  that  the  observation  of 

f , ,  ...,  f  reveals  no  statistical  information  about  y. 

I  n 

Let  x  be  a  random  vector  defined  on  the  basic  probability  space  (ft,  F,  p) 

[3] 

x  is  called  a  Gaussian  random  vector  if  it  has  the  distribution  law. 


MA)  =  — = “T  f 

j  2Ttf !?  }  xeA 


exp  -  (x  -  m) '  Z  ^ (x  -  m)  cb<  (2.2.14) 


where 


m  k  E{x)  = 


x(w)y (dm) 


£  =  E{  (x  '•  m)  (x  -  m) 1 } 


■1. 


(2.2.15) 


(x  (w)  -  m)  (x  (u)  -  m)  '  y  (dm) 


m  is  called  the  mean  or  expectation  of  the  random  vector  x,  and  £  is  called 
the  covariance  matrix  of  the  random  vector  x.  From  (2.2.14),  we  see  that 
the  statistical  law  of  a  Gaussian  random  vector  is  specified  completely  by 
its  mean  and  covariance.  We  shall  always  denote  a  Gaussian  vector  with  mean 
m  and  covariance  £  by  the  symbol  Q (m,  Z) . 


Two  Gaussian  vectors  x^,  x^  are  independent  if  and  only  if 
e{xx  2^2  M  {  — 1^—2^ 11  =  x1(a))p(dw)|  x£(a>)u(du) 

=  E{xx}  •  E)x£( 


[3] 


(2.2.16) 


Let  x(t) ,  t  e  [tg,  T],  be  a  random  n-vector  process  defined  on  the 
probability  space  (ft,  F,  u).  sc (, t )  ,  t  t  [t^,  T],  is  called  a  Gaussian  random 
n-vector  process  if  for  any  finite  set  {t^ . t  },  t^  e  [0,  T]  the  vector 

r  x(tx,  w)*i 


x(u>)  = 


L*(»v  w)J 


is  a  Gaussian  random  nm-vector. 

Another  observation  statistic  which  we  shall  consider  in  the  later 
chapters  is  the  "Gaussian  White  Noise  Process."  Different  interpretations 
of  this  kind  of  process  are  available.  One  may  view  it  as  a  formal  deriva¬ 
tive  of  a  Wiener  Process, ^  or  as  a  generalized  random  process ^  where  the 
observation  space  is  the  set  of  linear  functional  on  the  class  of  test 


with  statistical  laws: 
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i{0)  ±{j} 

£(k)  '* G  (0,  2(h))  ;  k  =  0,  1,  ... 


(2.3.2) 


Free  (2.3.1) v  sices  A(k)  is  licear  craasfocation,  x(k)  is  also  a  Gaussian 

m 

vector,  k  =  0,  1,  —  Let  f  be  seas  statistic  on  t,  F,  -) ,  and  let 

FCf)  denote  cite  r-algebra  by  f.  Suppose  that  f  is  independent  of  £{i), 

1  *  i,  ic  t  1,  — ;  then  for  i  i  k,  by  (2.2.12),  ve  have. 

Hi  *  X;£)  =  A<i)*(iSf)  a.s.  ;  x(i;f)  £  E{x(i) !?(£)}  (2.3.3) 


and 


£(i  +  l«f)  -  A(i)2a:f)A*(i)  V  E(i)  a.s. 


f(i!f)  =  H?x(i>x'(i)  F(f)» 


(2.3.4) 


Using  (2.3.4)  and  (2.3.3),  the  conditional  covariance  of  x(i)  relative  to 
?(?!,  denoted  by  I_-(i5f),  will  satisfy 

ZC(i  +  i|f)  =  A(i)IC{i|f)A'(i)  *  R(i)  a.s.  .  (2.3.5) 


In  addition,  if  the  conditional  distribution  of  x(k)  relative  to  F(f)  is 
Gaussian,  then  for  a>l  i  i  k,  3c{i)  is  a  conditional  Gaussian  vector  relative 
to  F(f).  The  statistical  information  of  the  statistic  f  is  contained  in 
F(£) ,  but  the  necessary  statistical  information  of  f  about  the  uncertainty 
of  the  future  state  of  the  system,  x(i),  i  2  k,  is  contained  in  the  con¬ 
ditional  distribution  of  jc(k)  relative  to  F(f);  and  if  it  is  Gaussian, 
k(kjf)  and  I^kjf)  completely  specify  the  conditional  distribution  of  x(i), 
i  £  k,  relative  to  F(f).  This  is  also  referred  to  as  the  Markov  property. 
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In  the  above  discussion,  the  observation  statistic  is  completely 
general.  If  the  observation  statistic  is  linear  in  x(j),  j  =  0,  1,  — ,  k, 
and  some  other  Gaussian  vectors,  e.g.,  f  =  (y(0) ,  y(k)},  and 

£(i>  =  C(i)  r.(i)  -^n(i)  i  -  0,  1,  ....  k  (2.3.6) 

vhere  ^(i)  is  F-measurable ,  independent  of  £(j),  j  —  k  and  of  Gaussian 

statistical  law,  i  =  0,  1- _ ,  k;  then  x(k)  is  conditional  Gaussian  rela-^ 

tive  to  F(f).^  There  inay  be  ocher  kinds  of  statistical  observations 
which  will  induce  a  conditional  Gaussian  lav  on  Jc(k),  but  in  this  thesis,  ve 
shall  only  consider  observation  statistics  of  the  type  given  by  (2.3.6). 

Consider  a  continuous  linear  stochastic  system  described  by 
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We  shall  always  assume  chat  x(tg)  and  JLC1),  i  £  [t^,  tj,  are  independent 
for  all  t  >  Cq.  Therefore  ic(t^)  is  independent  of  C(t),  t  e  [t^,  t],  for 

all  t  >  t^.  We  can  find  the  solution  of  (2.3.11),  x(t),  t  e  [t^,  T],  by 

successive  approximation:  t  e  [t^,  T] 

Xg(t)  =  x  )  i  ^(t)  =  xCtj)  +  [  A(*)x0(x)dT  +  j  C.(r)dT  (2.3.12) 

'C1  C1 

and 

x  (t)  =  x(t_)  +  f  A(i)x  .(T)d-  +  [  Ut)dT  n  =  1,  2,  ...  .  (2.3.13) 

_jj  1'  j  -  —n-x  J  - 

1  1 

By  the  assumptions  on  A(*)  and  R(*)>  this  procedure  will  converge  with 
x^(t)  -x(t),  t  £  [t^,  T]  a.s.,  and  x(t)  satisfies  (2.3.11). 

Let  f  be  an  observation  statistic  such  that  F(f)  and  F(^(t).  i  e  [t^,  X)) 
are  independent.  Suppose  that  the  conditional  distribution  of  x Ct^)  rela¬ 
tive  to  f  is  Gaussian.  Then  from  (2.3.12)  and  (2.3.13),  we  see  that  the 
conditional  distribution  of  x^(t)  for  a  fixed  t  t  [t^,  T]  is  Gaussian  rela¬ 
tive  to  f,  n  =  0,  1,  ...,  thus  ,  for  a  fixed  t  e  [t.,  T],  which  satisfies 
(2.3.11)  is  also  conditionally  Gaussian  relative  to  F(f).  Therefore  the 
complete  statistical  law  of  jc(t)  relative  to  f  is  described  by  its  condi¬ 
tional  mean  and  covarianca. 

From  (2.3.13),  we  see  that  for  all  n  =  1,  2,  _ 

ft  f'l  f~n-l 

lF(f)[  =  }l+|  A(t1)J  A('2)...|  A(Tn>dTn...dT]}-E{x(t1)  |F(f)}  a.s. 

Jt1  t2  '  tx 


(2.3.14) 
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Ejx  (t)x' (t)  F(f)  }  = 

*— n  — n  1 

r'l  rT  n-1 


A(T2)...j  Af^dV..dT1).Ej£(r.n)i,0:l)  F(f)[- 


'“I  =1 


'1 

•”n-l 


•t  r  1  f  o-l  .  fc 

(l+;  A(7.)  A(t  )..j  A(-  )dT  dx  )+  R(T)dT 

'C1  C1  '  C1 


f°»-l 


+  ’  A(C  )i  A  (c2)...|  A(cnl)  *  R(cn)dCn..-dCi 

'C1  'C1  'C1 

.  ft  -91  fCo-2  fn-1 

+(l  A(cl}  A/,c2)...;  A(=n_l>;  R(Cn)dcp...da1j  a.s. 


*  r  *  r 

i  ] 


(2.3.15) 


Since  x  (t)  -  :<(t)  a.s.,  £{x  (t)  F(f)}  -  £{x(t)  F(f);  a.s.,  and 
— n  —  — n 

E'x  (t)x’(t)  F(f);  -  £{x(t)x' (c)  F(f ) :  a.s.  Equations  (2.3.14)  and  (2.3.15) 
— n  — n 

imply  that  as  n  -  E{x(t)  F(f)}  and  £{x(t)x' (t) j F(f) }  satisfy  (a.s.): 


i(t!F(f))  =  A(t)x(t!F(f))  t  >  tL 


x(t'F(f))  i  E{x(t);F(f)}  (2.3.16) 


|(t  F(f))  =  A(r)f(t!F(f))  +  i(t;F(f))A*(t)  +  R(t)  t  >  tx  ; 

!(tfF(f))  £  Eix(t)x' (t)iF(f)}  .  (2.3.17) 

c 

The  conditional  covariance  of  >;{t),  t  -  t^,  denoted  by  Z  (t,F(f))  will  then 
satisfy 

ZC(t  j F(f ) )  =  A(t)ZC(tiF(f})  +  IC(c|F(f))A’ (t)  +  R(t)  a.s.  ,  t  i  Cj  .  (2.2.18) 

In  the  above,  the  observation  statistic  is  completely  general.  If  the 
conditional  distribution  of  _x(c-i )  relative  to  f  is  not  Gaussian,  then  x(t) , 
t  2  t^,  will  not  be  Gaussian  for  any  fi:ed  t;  however  the  conditional  mean 
and  covariance  of  x(t),  t  2  t^  are  still  given  by  (2.3.16)  and  (2.3.18). 
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In  this  thesis,  we  shall  assies  that  the  observation  statistic  is  of  the 


v(t)  =  C(t)  x(t)  -5-  n(t)  ,  t  e  [tg,  T  ] 


where  sj(t),  £  £  [cn,  T]  is  Gaussian  white  noise  with 


(2.3.19) 


E\l  2  n(t)dr  0 

t2  t2  *  .z2 

E{(|  n(’.)dt)([  ji(T)dx)  f=  [  0(t)dr 

t,  t. 


(2.3.20) 


(2.3.21) 


and  r?(t),  t  £  [t^,  Tj  is  independent  of  _£(t) ,  t  £  {t^,  T],  and  x(tg).  Kith 
such  observation  statistic,  we  see  that  ?  =  ?(Z.(7)>  7  £  lcr»  c,])  is  inde- 

JL. 

pendent  of  £(?),  t  z  ft..,  T] ;  furthermore  x^(t.  !P  )  is  F  -measurable  and 

'1  A  ti  tj 

x(t^|Fc  )  is  conditionally  Gaussian  if  x(tg)  is  Gaussian. If 

t_  <  t,  <  . . .  <  t  ,  we  have 

0  12  n 

F  ...  C  F  £  F 

t.  t_  t 

12  n 

In  the  more  general  nonlinear  case,  the  system  is  described  by 


x(t)  “  f.  (t,  x(t))  +  jj.(t) 


(2.3.22) 


where  s.(t),  t  z  [t^,  T},  is  a  Gaussian  white  noise  with  statistical  law 
(2.3.9),  (2.3.10),  and  £(t,  x(t))  is  F(£(r),  t  e  [t^,  t J) -measurable,  the 
solution  of  (2.3.22)  is  defined  as  the  process  which  satisfies 

rt  rt 

x(t)  =  x(Cq)  +  j  £_(t,  x(r))d~  +  C.(T)dx  a.s.  t  e  [tQ,  T]  .  (2.3.23) 

0  0 

If  f(t,  •)  satisfies  the  Lipschitz  condition 
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'  x-l)  -  *('•  &)  '  'j;  *1  "2  •  *1>  -*-2  £  R"  <2'3-24) 

where  a  is  some  constant;  then  the  method  of  successive  approximation  by 
setting  x(t)  =  x(tg)  and 

ft  -t 

x^U)  =  x(tQ)  +  j  £(T,  xn_1(T))d-  +  J  S/OdT  n  =  1,  2,  ...  (2.3.25) 


will  converge  almost  surely  to  x(t)>  as  n  -*■  ®. 1  J  The  interpretation  we 
used  here  is  Ito's;  the  reason  for  adapting  this  interpretation  is  due  to 
the  rich  mathematical  properties  one  can  deduce  and  utilize  by  using  this 
interpretation.  Ito's  theory  in  stochastic  differential  equation  will  not 
be  considered  in  here,  the  detail  can  be  found  in  [1],  [7],  [8]. 

Let  2c(t)  be  a  process  described  by  (2.3.22)  or  (2.3.23)J  :c(t)  is 
called  a  diffusion  processP^ '  ^  Let  C(*,  •)  be  defined  on  T  xRn  with 
real  scalar  value,  such  that  (^(t,  x) ,  Ct<t,  x) ,  and  (t,  x)  are  defined 
and  continuous.  The  differential  generator  of  x  with  respect  to  C  is  de¬ 
fined  by 


£(C(t,  x))  h  lim(s  -  t)  E{C(t  +  At,  x(t  +  At))  -  C(t,  jc(t)  |x(t)  =  x} 
s  +  t 

r n i  r o i  (2.3. 26) 


If  x(t),  t  e  [tg,  T],  satisfies  (2.3.22),  then 


[7], [8] 


£(C(t,  x))  =  \  trj R1/2(t) rC  (t,  x)R1/2(t)|  +  f(t,  x)'C  (t,  x)  .  (2.3.27) 


If  in  addition, 


|C|  +  ICJ  +  |x||cj  +  ixj2|c^.x|  1  k(l  +  ixj2  j  ;  (t,  x)  e  T 


(2.3.28) 
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then 


rc 

C(t,x(t))  =  C(t1>x(t1))+  i  [£(C(i,x(t))  +  C^(t  ,x(t))  ]cJt 

- 

rt 

Cx(T,x(x)),_£(T)dT 


(2.3.29) 


where  now  the  last  integral  must  be  interpreted  in  the  sense  of  Ito.^^ 
Let  be  the  sub-o-algebra  which  is  independent  of  F(£(r);  t  e  [t^,  T]). 
Since 


Elj  ^(t,  x(x))’  C(T)dTlF1  J  =  0  (2.3.30) 

C1 

i  f 171  f  Al 

we  have  from  (2.3.29)  the  Ito  s  integration  formula:1  i 

E|c(t,2S(t))!F1}  =  E{c(t1,x(t1))lF1}  +  e||  [£(C(t,x(t))  +  Ct (t ,x(t)]  di  | Fj_  }  . 

C1  (2.3.31) 

2.4  Stochastic  Optimization 

In  this  section,  the  mathematical  tools  for  stochastic  optimization 
^wbleras  are  stated,  and  the  outline  of  the  proofs  will  be  given.  These 
stochastic  optimization  techniques  will  be  used  in  later  chapters  to  solve 
different  stochastic  control  problems. 

Since  we  shall  be  considering  linear  systems  with  Gaussian  disturbances, 
the  process  which  we  shall  control  will  be  Gaussian.  Thus  an  adequate  de¬ 
scription  of  the  process  is  the  evolution  of  its  mean  and  covariance.  As  a 
result,  we  shall  deal  with  a  set  of  deterministic  equations  which  describes 
the  "trajectory"  of  the  mean  and  covariance.  In  many  cases,  we  can  transform 
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♦ 


a  linear  stochastic  control  problem  into  a  deterministic  control  problem 
where  the  dynamics  of  the  deterministic  system  are  described  by  a  set  of 
matrix  and  vector  differential  equations.  After  making  such  transformation, 
the  technique  cf  the  matrix  minimum  principle  can  be  used  to  obtain  neces¬ 
sary  conditions  for  optimality ,  ^  in  the  following  way. 

Discrete  Time  Control  Problem: 

A  set  of  matrix  and  vector  difference  equations  is  given: 


X(k  +  1)  -  X<k)  =  F(k,_X(k),x(k),U(k)) 
2«(k  +  1)  -  x(k)  =  f (k,x(k) ,X{k) ,U(k)) 


k  =  0,  1,  ...»  H  -  1 
X(*)  11  2^)5  =  Xq 


(2.4.1) 


with  U(k)  c  S,  constrained  control  set,  X(k)  £  M  ,  x(k)  £  RP.  Consider  the 
~  —  nm  — 

scalar  cost: 


N-l 

J  =  K(X(N) ,x(N) )  +  ^  L(k,U(k),X(k),x(k))  .  (2.4.2) 

k=0 


It  is  assumed  that  F(k,*),  f_(k,*)>  K(- )  and  L(k,*)  satisfy  the  conditions 

[33] 

required  by  the  discrete  minimum  principle.  The  control  problem  is  to 

* 

choose  U  (k) ,  K  =  0,  ...»  N  -  1,  such  that  the  cost  (2.4.2)  is  minimized  sub- 
ject  to  the  constraint  (2.4.1)  and  (k)  e  S,  k  =  0,  . ..,  N  -  1. 

Define  the  Hamiltonian  function 


H(X(k),x(k),P(k+l),£(k+l),U(k))  k  L(k,U(k) ,X(k) ,x(k)) 

+  f  ’  (k,x(k)  ,X(k)  ,U(k))£(k+l)+tr{F(k,X(k)  ,x(k)  ,U(k))P'(k+l) } 


(2.4.3) 


where  P(k) ,  £(k)  are  the  costate  associated  with  X(k)  and  x(k)  respectively. 
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Theorem  2,4,1:  (Matrix  Minimum  Principle:  Discrete  Time) 

*  *  * 

Let  IJ  (k),  k  =  0,  ....  N  -  1  be  the  optimal  control  and  X  (k) ,  x  (k) , 

k  *  0,  . ..,  N  be  the  optimal  state?  then  there  exists  a  costate  matrix 

*  *  k 

£  (k)  associated  vitf  X  (k) ,  and  a  costate  vector  £  (k)  associated  with 

* 

7C  (k)  such  that  the  following  relations  hold: 

1)  Canonical  Equations : 


*  *  3H 

X  (k  +  1)  -  X  (k)  = 


3P(k  +  1) 


;  x  (k  +  1)  -  x  (k)  = 


3£(k  +  1) 


*  *  3H 

P  (k  +  1)  -  P  (k)  =  - 


3X(k) 


;  p  (k  +  1)  -  d  (k) 


3H 


3x(k) 


(2.4.4) 


2)  Boundary  Conditions: 


X  (°)  -  Xq  ;  x  (°)  =  Xq 


(2.4.5) 


£*<«  -  jgflLffltriLOO)  J  /(„)  =  (2.4.6, 

3X  (S)  3x  (S, 


3)  Minimization  cf  the  Hamiltonian: 

For  every  U  e  S,  and  for  each  k  =  0,  1,  . ..,  N  -  1 

h(x*00  ,x*(k) ,P*(k+l)  ,£*(k+l) ,U*(k))  <  H(x*(k) ,x*(k) ,P*(k+l) ,£*(k+l) ,u) 

(2.4.7) 

Continuous  Time  Control  Problems: 

A  set  of  matrix  and  vector  differential  equations  is  giver.: 


x(t)  =  F(t,x(t),x(t),u(t)) 

x(t)  =  .f(t,x(t)  ,x(C)  ,u(t)) 


Xf0)  = 

"  *o 


(2.4.8) 
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with  U(t)  t  S,  constrained  control  set,  X(t)  t  M  ,  x(t)  c  R  .  Consider 

nm  — 

the  scalar  cost: 


fT 

J  =  K(X(T),x(T))  +  |  L(t,X{t),x(t),U(t))dt  ;  T  fixed  .  (2.4.9) 


The  usual  differentiability  conditions  for  £(•)>  _f(*)»  K(*)»  and  L(‘)  are 

& 

assumed  to  be  satisfied.  The  control  problem  is  to  choose  U  (t),  t  t  [t^,  TJ, 
such  that  ."he  cost  (2./,.9)  is  minimized  subject  to  the  constraint  (2.4.8) 

: k 

and  U  (t)  c  S. 

Define  the  Hamiltonian  function 


H(X(t),x(t),P(t),£(t),U(t))  4  L(t,X(t) ,x(t) ,U(t))  + 


f 1 (t,x(t) ,X(t) ,U(t))p(t)  +  tr{F(t,X(t) ,x(t) ,U(t))P* (t)} 


(2.4.10) 


where  _P(t),  £(t)  are  the  costate  associated  with  X(t)  and  x(t)  respectively. 

Theorem  2.4.2:  (Matrix  Minimum  Principle:  Continuous  Time) 

Let  IJ  (t),  t  e  [t^,  T],  be  the  optimal  control  and  X  (t),  x  (t), 

Jc  £ 

t  e  [t^,  TJ,  be  the  optimal  state,  then  there  exist  costates  P  (t),  £  (t) 
such  that  the  following  conditions  hold: 

1)  Canonical  Equations: 


X  (t) 


9P(t) 


.*  ,  3H 

J  *  (t)  =  -r~y 


3£(t) 


(2.4.11) 


P*(t)  „  _  _iii_ 

-  3X(t)  * 


’  £  (t)  “  "  3x(t)  * 


(2.4.12) 
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2)  Boundary  Conditions: 

xVqJ-Xq  ;  x*(t0)  =  x0  (2.4.13) 

P*(T)  ,  ;  £*(T)  (2.4.14) 

8X  (T)  3x  (T) 

3)  Minimisation  of  the  Hamiltonian: 

H(x*(t)  ,x*(t)  ,P*(t)  ,£*(t)  ,U*(t)j  <  H(x*(t),x*(t),P*(t),£*(t),U  J  (2.4.15) 
for  all  IJ  e  S  and  t  e  [t^,  T]. 

The  matrix  minimum  principle  (both  discrete  and  continuous)  is  a  just  straight¬ 
forward  extension  of  the  vector  minimum  principle,  Holtzman  and  Halkin 
[33],  Pontryagin,  et  al.  [11].  Theoretically,  the  justification  of  the 
matrix  minimum  principle  hinges  on  the  existence  of  a  mapping  from  Mnm  to 
RPm.  The  details  were  carried  out  by  Tse  [9];  see  also  Athans  [12]. 

The  matrix  minimum  principle  only  provides  us  with  necessary  conditions 
for  optimality.  A  control  and  its  corresponding  state  trajectory  which 
satisfies  the  matrix  minimum  principle  will  be  called  extremal  control  and 
extremal  state  trajectory.  If  one  can  prove  the  existence  of  optimal  con¬ 
trols  and  the  uniqueness  of  extremal  controls,  the  matrix  minimum  principle 
also  served  as  a  sufficient  condition  for  optimality.  But,  in  general,  the 
matrix  minimum  principle  does  not  provide  sufficiency.  It  will  be  convenient 
if  one  can  have  some  sufficient  conditions  for  optimality,  so  that  one  can 
easily  test  to  see  whether  an  extremal  control  is  optimal  or  not.  It  turns 
out  that  to  look  for  sufficient  conditions,  it  is  often  easier  (and  more 
general)  if  we  consider  the  original  stochastic  control  problem  without 
transforming  it  to  deterministic  description  in  terms  of  mean  and  covariance. 


# 


Discrete  Time  Stochastic  Control  Problems: 

A  discrete  time  stochastic  process  is  described  by 


jc(k  +  1)  =  £(k,x(k)  ,ii(k) )  +  C.(k)  k  =  k^,  k^  +  1,  ...  (2.4.16) 

with  x(k)  e  Rn,  u(k)  r.  RT.  Let  x(0) ,  C.(k) ,  k  =  0,  1,  ...  be  independent 
Gaussian  Vectors  defined  on  the  basic  probability  space  with  statistical 
law  (2.3.2). 

Let  U(kQ,  k)  4  {j±(k0),u(k0  +  1) , . . . , u(k) }  denotes  the  control  sequence, 
and  g(kQ),  g(kQ  +  i,u(kQ)),  g(kQ  +  2,U(kQ,k0  +1)),  ....  g(k,U(kQ,  k  -  1)), 

...  is  a  sequence  of  observation  statistics  which  depends  on  control  se¬ 
quence,  such  that  for  all  control  sequences  F(k,U(kQ,k  -  1)  CF(k  +  i,U(kQ,k)), 
where  F(g(k,U(kQ,k  -  1))  4  F(k,U(kQ,k  -  1)).  Let  {x^  k-l)^k^k=k  be  the 
process  described  by  (2.4.16)  when  control  sequence  U(ky,N  -  1)  is  applied. 
Assume  that  x^^  ^.-i)  (k)  is  F(g(k,U(ky,k  -  1) )-measurable  when  the  control 

is  restricted  to  be  of  the  form; 


u(k)  =  £(k,g(k,U(k0>k  -  1))  c  S  (a.s.) 


(2.4.17) 


The  control  problem  is  to  find  air  optimal  control  law  (k,  U  (k,Q,k-l)) 
such  that 

N-l 

J(U(kQ,N  -  1) j F (g (kQ) )  =  E{K(x(N))  +  V  L(k,x(k),u(k))!F(g(k0))f  (2.4.18) 

k=kQ 

is  minimized  subject  to  (2.4.16). 

Theorem  2.4.3:  (Optimality  Criteria:  Discrete  Time; 

*  n 

Suppose  that  there  exists  a  control  strategy  £  (i,*):  Ir,  M  ,  iXR  -*•  S 

n  i 

and  a  scalar  function  I fl.  ,xR  •*  R  such  that  almost  surely v 

L “o  »*'■*”•**  J 


1)  C(N,x)  =  K(x) 


(2.4.19) 


I 
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2)  0  =  EiL(k,x,f  (k,x))^C(k+l,x  (k*l,x)  F(k,U(k0,k-i));-C(k,x) 

i  E{L(k,x,u)-rCCk-rl.x  (k^-l,x)  |F(k,U(k  .,k-l)  }-C(k,x)  a.s. 

u  U 


(2.4.20) 


k  =  kQ,  kQ  +  1,  ... 


where  for  k  =  k_,  k.  -5-  1,  — ,  X  -  1 
0  u 


x  (k  i,x)  =  A(k)x  +  B(k)o  (k,x)  -*■  ^(k) 


(2.4.21) 


x^k  +  l,x)  =  A(k)x  +  B(k)u  +  ^(k)  ;  u  =  £(k,gj.)  £  S 


i.e.,  u  is  any  pernissible  control  value  at  k.  Than  the  control  law  i  (*,*) 

is  optimal  and  C(kg,x(kg))  is  the  optimal  cost  a.s. 

* 

Proof:  Let  x  (k),k  =  k^, _ ,N  be  the  random  vectors  which  satisfy  the  dif¬ 

ference  equation  (a.s.) 


x  (k  +  1)  =  A(k).x  (k)  +  8(k)_;  (k,x  (k))  +  £(k)  ; 


x  (^q)  =  x(kg) 


(2.4.23) 


By  using  lemma  2-2.6,  we  have  from  (2.4.20)  that:  (a.s.) 

C(kg,x(kg))  =  E{L(kQ,x(k0) ,_o*(k0,x(k0))+C(kc+l,x  (k+1) | F (g (kQ) ) } 

=  E(L (kg , X (kQ) )±  (kg , x (kQ) ) I F (g (kQ) ) } 

+E{ E{ L (kQ+l , x* (kQ+l) )±* (kQ+l ,x* (kQ+l ) ) 

+C (kQ+2 , x* (kQ+2) ) [ F (kQ+l  ,±  (kQ ,g (kQ) ) } | F (g (kQ) ) } 
k°+1 

=  E  ^  L(k,x*(k),6^  (k,x  (k)) |F(g(kQ)) 
k=k0 

+E{C(kQ+2,x  (kQ+2) ) i F(g (kQ) ) } 


(2.4.24) 
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Inductiveiy  and  keeping  (2.4.19)  in  dad,  we  have 

K-l 

C(k0,jc(k0))  =  E{K(x*(S))^  L(k,x*(k>,o*{k,x*(k)>iF(g(k0»|  a.s.  (2.4.25) 

k=kQ 

Xow  let  U  =  :u  (i) be  ..ay  admissible  control  law  of  the  fora  (2.4.17). 

Let  x  (k) ,  k  =  k^,  +  1,  ....  S  be  the  random  vectors  which  satisfy  (a.s.) 

x°(k-f  1)  »  A(k)x°(k)  +3(fc)u°(k)  +  C(k)  ;  x^Ot  )  =  x(kQ)  .  (2.4.26) 

By  (2.4.17),  u°(k)  is  F(k,U°(k^,k  -  Immeasurable .  Using  lemma  2.2.6  and 
the  inequality  of  (2.4.20),  we  nave  (a.s.). 


CCkQ,x(k0))  I  E{L(k0,x(k0) ,u°(k)4C{k0+i.x°(k0+l)> :F(g(k0)) } 

-  E{L (k0 , x  (kQ)  )  ,  u°  (k)  ;  F(g  (k0)  )  }tE{  e{  L (k0'l , x°  (kQ+l) , u°  (k0-fl)  ) 
+C (k0+2 ,  x° (k0+2) ) 5  F(k0+1 ,u° (k0) ) } 1 F (g (k0) ) } 


kn+1 


=  E {  ^  L  (k , x°  (k)  ,  u°  (k)  )  !  F  (g  (kQ)  ))  +r{ c  (kQ-f 2 ,  x°  (kQ-i-2)  )  j  F (g (kQ)  )  }  . 

k_k0  (2.4.27) 

Inductively  and  using  (2.4.19),  we  have 


N-l 

C(k0,x(k0))  4  E{K(x0(N))  +  ^  L(k,xO(k),u°(k))iF(g(k0))}  a.s.  (2.4.28) 

k=kQ 

Combining  (2.4.25)  and  (2.4.28)  we  have  the  assertion  of  the  theorem. 
Continuous  Time  Stochastic  Control  Problem: 

A  continuous  time  process  is  described  by 


x(t)  =  jE(t,x(t))  +  B(t)u(t)  +  £(t) 


(2.4.29) 


where  x(tj  ■  R  J_(t.  )  satisfies  the  Lipschitz  condition  (2.3.18). 

and  _z(:)»  T  --  I Cq ,T j  is  white  Gaussian  noise  with  statis¬ 
tical  law  (2.3.9)  and  (2.3.10).  Denote  the  control  U{tQ,t)  =  *,u(t), 

:  -  iC^,t)|.  Let  g(t,Uitg,t))  be  an  observation  statistic  such  that  if 
£i  '  s2*  ?(t1»ulc0,t))  =  F(g(t()U[c0,c1))  C  F(g(t7*L*ftort^»  i  F(t7Ult0,t9)); 
at  t  *  tg,  g(t0,Uft0,t0))  =  g(tg)  ana  is  independent  of  control.  Let 
'•3yfc  ,.j(t)»  c  Icq»T] ’*  be  the  process  described  by  (2. A. 29)  when  Uft^.T) 
is  applied.  We  assure  that  x^.^  cj(c)  is  p(c»^ttQ,t))-seasurable  wnen  c^e 
control  is  restricted  to  be  of  the  fora 


u(t)  =  at,g(c,t’[t  ,t)))  *  S  (a.s.) 


(2. A. 30) 


The  control  problen  is  to  find  optimal  control  law  of  the  fora  (2. A. 30) 
such  that  the  cost 

(  fT  1 

J(U(t0,t])|F(g(t0))  =  E|K(x(T)  +  ;  L(t,x(t),u(t))dt!F(g(t0))  J  (2. A. 31) 

‘‘  co 

is  niniaized  subject  to  (2.4. 29) . 

For  a  fixed  control  U  (tg,T)  of  the  fora  (2. A. 30),  we  have  a  fixed 
diffusion  process  described  by 


x(t)  =  f(t,x(t))  +  B(t)u  (t)  +  C(t) 


(2. A. 32) 


and  we  can  associate  with  U  ( ,T ]  a  fixed  differential  generator  £  q(0 

u 

Let  C(t,x_)  be  a  scalar  function,  we  have 


£  (C(t ,x)  )  =  tr  {R1//2(t)C  (t,x)R1^2(t)  }4-(X(t,x)+B(t)u°(t)) 


(2. A. 33) 


Theoren  2.4.2:  (Optimality  Criteria:  Continuous  lime) 

*  n  .  * 

Suppose  there  exists  a  control  law  (*,*):  It^.T)  xR  —  S  with  (t,*) 

n  1 

satisfying  the  Lipschitz  condition,  and  a  function  C (*,*) :  ltg,T)  xR‘  -  R 
such  that 

1)  C(t,x;,  C^t.x),  (^(t,::)  are  continuous  and  for 

some  k 

C  +  ,C  :  +  jxjtcj  +  ixi2!^  <  k(l  -b  ;x!2)  (t,x)cIt0,T)xRn  (2. A. 34) 

2)  C(T,x)  -  K(x)  a.s.  (2.4.35) 

3)  0  =  Ct(t,x)  +  £{  £Q*(C(t,x)  +  L(t,x,i*(c,x))|F(t,U[t0,t))} 

-  Ct(t,x)  +  E{  £p(C(t,x)  +  L(t,x,^(t,x)) }F(t,U[t0,t))}  a.s.  (2.4.36) 
for  all  (t,:c)e  [tg.TjxR11,  and  £(t,*)  satisfies  Lipschitz  condition. 

* 

Then  _i  (t,x(t))  is  the  optimal  control  law  and  C(tQ,x(tg))  is  the  optimal 
cost  a.s. 

iz 

Proof :  Let  x  (t)  be  the  resulting  diffusion  process  described  by  (2.4.32) 

* 

where  we  adapt  control  law  _£  (*,*)•  Using  the  Ito's  integration  formula 
(2.3.31)  applying  to  the  process  x  (t),  we  have 

E<C(t0,x*(t0))|F(g(t0))}  =  C(t0,x*(t0))  =  E{C(T,x*(T))|F(g(t0))> 

ffT  * 

-  E(j  U0*(C(t,X  (T») 

co 

+  Ct(t,x* (T))]dT |F(g(t0>) j  a.s.  (2.4.37) 

By  lemma  2.2.6  and  Equation  (2.4.35),  (2.4.36),  we  obtain 


C(c0,x  (t0)) 


=  E-E:C(T,x  (T) )  •  F(x  (T))'*;F(g(r.0))> 


-e|e||  I  £^*(C(-.,x  (*))) 


Ct(t,x  {:)) ]dt  F(t,U[c0,c»  F(g(tQ)) 


=  E  K(x*(T))  -r  L(:  ,/(t)  ,£*(:, x*(T))d:F(g(t0)) 


(2.4.38) 


Let  _;(t,*)  be  any  control  law  which  satisfies  the  Lipschitz  condition,  and 
x°(t)  be  the  resulting  diffusion  process  described  by  (2.4.32)  when  control 
law£°(t,*)  is  used.  The  Ito's  integration  formula,  applied  now  to  the 
process  x  (t) ,  gives  us 

•T 

C(t0,x°(t0))  =  E{C(T,x0(T))'F(g(t  ));  -  E{!  [£  0(C(r ,x°(t))) 

'  C0 

+  C.(T,X°(T))]dT:F(g(t0))  } 

-T 

i  L|K(x°(T))  +  ;  L(T,x°(T)2C(T,x(T))dTiF(g(t0))  }  a.s. 

'  co 

U  (2.4.39) 

where  the  last  inequality  comes  from  the  inequality  part  of  (2.4.36),  also 
lemma  2.2.6  is  being  used  in  deriving  (2.4.39).  Note  thatc(*,*)  is  de¬ 
fined  on  [tg,T)xRn;  now  equations  (2.4.38)  and  (2.4.39)  yield  the  statement 
of  the  theorem. 

2.5  Generalized  Matrix  Riccati  Difference  Equations 

.  00 

For  a  given  sequence  of  matrices  V  =  { V (k) },  ,  ,  V(k)  e  M  ,  k  =  k^., 

—  k=kg  —  nm  U 

OC 

k.  +  1,  ...,  let  {P  (k.k^jF)},  ,  be  the  sequence  which  satisfies  the  linear 
0  -v  0  -  k=kQ 


matrix  difference  equation: 
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P  (k+l,k  ;F)  =  (A(k)-V(k)D(k)A(k))P  (k,kn;F)(A(k)-V(k)D(k)A(k))’ 
v  u  —  —  —  —  —  v  u  —  —  —  —  — 


+CIn’V(k)D(k))ai(k)  (I^-V  (k)D(k) )  ‘+2,  (k)-^  (k)+V (k) R (k) V '  (k) 


l 


(2.5.1) 


where  A(k)  •:  M  ,  D(k)  c  M  are  bounded  uniformly  for  all  k.  We  assume 
—  nm  —  mn 

that  F,  C^(k),  (^9(k)  are  symmetric  nonnegative  definite  nxn  matrices  with 
0^(k)  i  £9(k),  k  =  kg,  k^  +  1,  ...,  and  that  R(k)  is  symmetric  nonnegative 
definite  mxm  matrix.  Since  (2.5.1)  is  linear,  therefore  for  any  arbitrary 

CO 

V(k)  c  M„m,  k  =  kg,  k^  +  1,  ....  'P^(k,k..  ;F) ;.  k  exists,  is  unique  and 

P^(k,k0;£)  -  0,  k  =  kQ,  kQ  +1,  ...  . 

When  V(k)  ranges  over  M  ,  k  =  k_,  k.  +  1,  . . . ,  we  generate  a  solution 

nm  U  0 

setfi^  =  •  (PyCk.k^jF))^^  V(k)  •:  M^,  k  =  kfi,  kn  +  1,  ...}.  All  elements 


O’  0 


in  the  solution  set  fi  is  a  sequence  of  symmetric  nonnegative  definite 

0 

nxn  matrices. 

Definition  2.5.1:  (Minimal  seq_»«-..ce)  Let  S,  =  {(M(k))°  ,  iM(k)  t  M  , 

ko  J  k"ko  “  nn 

M(k)  2  0,  k  =  k. ,  k  +1,  ...};  an  element  (M°(k))  e  £  is  called  a 
0  0  -  k«k0  k0 

minimal  sequence  with  respect  to  &.  if  for  all  (M(k))*  c  £  ,  we  have 
„  0  k=tc0  0 
M  (k)  l  M(k),  k  «  kQ,  kQ  +  1,  ...  . 

For  a  given  set  £  ,  a  rinimal  sequence  may  not  exist;  but  if  it 

0 

exists,  it  must  be  unique.  In  the  following,  it  will  be  shown  that  the 

solution  set  R  has  a  unique  minimal  sequence. 

0 

Let  us  define  the  matrix  Ik(V,P)  by  (k  =  kQ,  kQ  +1,  ...) 

4<V,P)  =  (A(k)-V  D(k)A(k))P(A(k)-V  D(k)A(k)) ’  +  (!  -V  D(k))Q.(k)(I  -V  D(k))’ 
— k  —  - —  — n - -^1  — n - 


+S2(k)-fi1(k)+V  R(k)V* 


(2.5.2) 


1 


where  V  e  M  ,  P 


nr.  -  nil 


:>:ed  P  M  ,  define  the  set 
-  nn 


i',  (P)  =  -V  ;  M  (*),  is  satisfied  -  where  the  condition  (*).  is  given  bv 
fc  —  ~  inn  k  k 

(*)2  ViR(k)+Dlk)  (k)+A(k)P  A’  (k})D’  (k)  •  -  fA(k)P  A’(k)  ^(kitUUk)  .  (2.5.  j  . 

We  have  the  following  lemma: 

Lemma  2.5.2:  (Minimum  property)  Let  ?  e  M  ,  and  ?  2  0;  if  V  si’,  (p)  , 

nn 

then  for  all  V  z  M  : 

—  nm 


ijjv,?)  1  ^(V.P)  ;  k  =  kQ>  kQ  +  1,  ... 


(2.5.4) 


Proof:  Let  us  denote 


R(k,P)  =  R(k)  +  D(k)(^i(k)  +  A(k)P  A’ (k))D* (k) 


k  -  kp ,  k^  +  1 ,  ... 


The  condition  (*)^  can  now  be  written  as 


V  R(k,P)  •«  (A(k)PA’(k)  -  Qi(k))  D’(k) 


k  =  V  ko  +  1*  ’ 


(2.5.5) 


Let  V  cl'  (P),  V  must  satisfy  (*)’;  and  so  for  \?  z  M  ,  we  have 


-  (V,P)+(V-V)R(k,P) (V-V) • 


A(k)P  A’(k)+21(k)-V  D(k) (A(k)P  A* (k)+fi  (k) ]- [A(k)P  A1  (k)-»$  (k)  ]D' (k)V’ (k) 
+2V  ^(k,P)^,+g2(k)~21(k)+V  R(k,P)V'-V  R(k,P)V*-V  R(k,P)V’ 

A(k)P  A’ (k)+21(k)+^2(k)-V  R(k,P)V,-(A(k)P(k)A’(k)+21 (k))D'(k)V' 


-VDtkl^W+ACkJPA'fk))  =  ^(V.P)  k  =  kQ,  kQ  +  1,  ... 


(2.5.6) 
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Since  R(k,_P)  0,  (2.5.4)  follows  from  (2.5.6). 

An  immediate  conseouence  of  the  lemma  is  that 


“  ^k(-2,-)  *  — 1  *— 2  C  Uk(-* 


(2.5.7) 


o  ,« 

Theorem  2.5.3:  There  exists  a  unique  minimal  sequence,  IP  (k,k„;F>;,  , 

u  fc-K0 

with  respect  to  the  solution  set  ft,  . 

0 

c  00 

Proof :  Let  us  construct  the  sequence  (k,k..;£)},  as  follows:  Set. 
^(k^.k^jF)  =  _F,  and  choose  V°  (k^)  c  L'^  (F) .  Such  a  V°(k^)  may  not  be 
unique,  but  by  (2.5.7)  and  (2.5.1),  this  gives  rise  to  a  unique 
P°(k  +  l,k.;n  =  -  (V°(k.),F),  V°(kn)  c  If.  (F). 

-o  o-  -k0-o--o  kQ- 

Assume  that  V  (k) ,  k^,  k.  +  1,  . ...  k^  +  i,  have  been  chosen  induc¬ 
tively  with  V°(k)  L',  (P°(k,k_;F)  and  a  unique  sequence 

K.  U  ”* 

P°(k+l,k0;F)  =  £k(V°(k),P°(k,k0;F))  ,  k  =  k0,k0+l,...,k0+i  (2.5.8) 

has  been  constructed.  Choose  V0(kg  +  i  +  1)  z  lf^  +i+1(P°(kQ  +  i  +  l.k^F)). 
By  (2.5.7)  and  (2.5.1),  this  gives  a  unique 


P°(k0+i+2,k0;F)  =  :ik  +.+1(V°(k0  +  i  +  l),P°(k0+i+l,k0;F))  .  (2.5.9) 


The  sequence  v P  (k,k.;F);  thus  constructed  is  unique. 

U  -  k-kQ 

,  oc 

Let  V  -  \V(k).'^_^  be  an  arbitrary  sequence  with  V(k)  s  M  ,  k  =  k^, 
kjj  *■  1,  ...  .  By  lemma  2.5.2, 


P.Vl.VD  *  l  (kn+l,kn;F) 


0  "’•'O’ 


(2.5.10) 


Assume  that  for  i  ^  1 , 


2  ^  (k0+i:ko''£> 


(2.5.11) 
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From  (2.3.1)  and  (2.3.2),  w 


'r  a  given  V  •  M  : 

-  nm 


^(V,?^  2  ^k(V,Py)  if  Px  -  F,  2  0  ,  k  =  k0,k0+l,...  .  (2.5.12) 


Combining  (2.5.1),  (2.5.2),  (2.5.4),  (2.5.11),  and  (2.5.12),  we  have 


P^+i+l.k^F)  -^o41(V(k0+i).Pv<koJ.l,koiF»  - 


4cg+i  S0+li  ^'r'o’rl,k0;-)) 


<Vi.koiF)) 

=  P°(k0+i,k0;F)  .  (2.5.13) 


The  theorem  follows  from  induction. 
Definition  2.5.4:  The  set  of  equations 


P(k+l,kn;F)  =  (I  -V(k)D(k))(A(k)P(k,k-;F)A'(k)+^.(k))(I  -V(k)D(k))' 
u  —  — n  —  —  —  —  u -  l  — n  —  ~ 

^2(k)-S1(k)+V(k)R(k)V,(k)  ;  P(k0,k0;F)  =  F 

(2.5.14) 

V(k)  (R(k)+D(k)  (A(k)P(k,k0;F)A'  (k)^  (k) )D’  (k)) 

=  (A(k)P(k,k0;F)A'(k)  •*  Qj  (k))  D’(k) 

is  called  the  generalized  Matrix  Riccati  Difference  equation,  and  the 
unique  solution  is  called  the  Riccati  sequence,  which  is  also  the  minimal 
sequence  with  respect  to  8  . 

ko 

The  above  definition  is  meaningful  because  of  theorem  2.5.3.  In  the 
special  case  when  R(k)  or  D(k)£^ (k)D' (k)  (or  both)  is  positive  definite, 
then  (2.5.14)  can  be  written  as  a  single  nonlinear  difference  equation: 
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P (k U ,k0 ; F)  =  A (k)P (k , kQ ;  F)  A  ’  (k)-tfi2  (k)- [A(k)P (k,kQ ; F)A '  (k)-^  (k)  JD'  (k)  • 

{R(k)+D(k)  (A(k)P(k,k0;F)A’  (kj^CKjjD*  (k)}-1D(k)  [A(k)P(k,k0;F)A'  (kJ+^Ck)] 

P(k0,kQ;F)  =  F  .  (2.5.15) 

[28]  r29] 

Equation  (2.5.15)  is  the  Matrix  Riccati  Difference  equation.  * 1 

2.6  Generalized  Matrix  Riccati  Differential  Equations 

Let  V(t)  he  arbitrary  bounded  measurable  nxm  matrix  defined  on  [t^,T]. 

Let  F  (t,trt;T)  be  nxn  matrix  defined  on  [t.,T]  which  satisfies 
— v  U  —  0 

^(t.tjjjF)  =  (A(t)-V(t)D1(t))Pv(t,t0;F)+Pv(t,t0;F)(A(t)-V(t)D1(t))! 

+V(t)^(t)V'(t)+(I  -V(t)D,(t))R(t)(I  -V(t)D.(t))’  ; 

—  — n  —  — l  —  — n  —  — z 

=1-0  (2.6.1) 

where  A(t)  is  nxn,  D^(t),  ^(t)  are  mxn;  R(t)  is  nonnegative  definite  nxn 

matrix  and  C[(t)  is  mxm  nonnegative  definite  matrix  (all  matrices  are 

assumed  bounded  measurable).  Since  (2.6.1)  is  linear,  the  solution 

P^^tgjF),  t  e  [ tg ,T] ,  exists  and  is  unique  for  a  fixed  bounded  measurable 

V(t)  (nxm)  defined  on  [ t:  , T ] . 

u 

When  V(t)  ranges  over  the  set  of  all  bounded  measurable  mxm  matrices 

T 

defined  on  [tn,T],  it  generates  the  solution  set  ft  =  {P  (t,tA;F), 

u  tp  — v  U  — 

t  e  [tQ,T]|v(t)  is  bounded  measurable  nxm  matrix  defined  on  [tg,T]}. 

Definition  2.6.1:  (Minimal  function)  Let  =  { M ( t ) ,  t  e  [ tn ,T] | M( t)  >  0, 

o  T  0 

t  r  [tg,T]].  An  element  M_  (  *)  e  #1  is  called  a  minimal  function  with  respect 

o 

to  if  for  all  M(-)  e  &  ,  M°(t)  <;  M(t),  t  e  [tn,T]. 

xo  T'o  “  ~  u 

Let  us  define 

A(t,V(t))  =  A  (t)  -  V(t)D  (t) 


(2.6.2) 
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The  solution  of  (2.6.1)  is  given  by^^ 

P^t.tQjF)  =  ±A(t,t0)F  ±i(t,tQ)  +  |  ^(t,T){V(T)^(T)V'(T)  + 

Z0 

t4  -  V(T)D2(T)]-R(T)[In-  V(T)D2(T)]*Hi(t,T)dT  .  (2.6.3) 

Since  I?,  R(x),  Q(x)  are  all  nonnegative  definite  matrices,  we  have 

Vt.VF)  *  0  ;  t  e  [tQ,T]  .  (2.6.4) 

Define  the  matrix 

I(t,V,P)  k  A(t,y)P  +  P  A' (t,V)  +  V  £(t)V*  + 

(4  "  V  D2(t))R(t)(In  -  V  D2(t))’  (2.6.5) 

where  V  is  bounded  nxm  matrix,  and  P.  is  bounded  nxn  matrix.  For  a  fixed 

-  e  Mnn’  define  the  set  k  (P)  »  {v  e  M  |  (*)  is  satisfied  where  (*)  is 
mi  c  nm  t  t 

the  condition 

<*>t  V(^(t)  +  D2(t)R(t)D'(t))  =  P  Dj(t)  +  R(t)Dj(t) 

Lemma  2.6.2:  (Minimum  Property)  Let  P  e  M  ,  and  P  >  0:  if  V  e  \i  (P) 

—  nn  —  —  —  t  —  ’ 

then  for  all  V  e  M  ,  we  have 
—  nm 

Kt.V.P)  i  i(t,V,P)  ,  t  e  [t0,T]  .  (2.6.6) 

^roof:  Let  e  (P.)  >  by  using  (*)  we  have 
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l(t,V,P)+(V-V)  (D2(t)R(t)D^(c)+g(t))  (V-V) ' 

=  A(t)P-V  D^OP+P  A’  (t)~P  Dj(t)V’+R(t)-V  D2(t)R(t)-R  D^(t)V' 

+2V(D2(t)R(t)D^(t)+£(t))V'+V(D2(c)R(t)D^(t)+g(t))V,-V(D2(t)R(t)^(c) 

+^(c))V,-V(D?(c)R(t)D^(t)+^(t))V’ 

=  A(t)P-V  Dj ( t ) P+P  A' (t)-P  D^(t)V'+R(t)-V  D2(t)R(t)-R  D^(t)V’ 

+V(D2(c)R(c)D^(t)+S(t))V'  =  l(t,V,P)  t  c  [C0>r]  .  (2.6.7) 

Since  R(t)  i  0,  Q(t)  t  0,  (2.6.7)  implies  (2.6.6)  immediately.  From  the 
lemma,  we  have 


X(t.VrP)  =  I(t,V2,P)  if  — i— 2  e  \  (— >  *  (2.6.8) 

Theorem  2.6.3:  There  exists  a  unique  minimal  function  P° ( t , t^ ;F) , 

t  e  [t0,T],  with  respect  to  the  solution  set  fiT  . 

C0 

Proof;  Let  us  construct  a  sequence  {^(t, tQ;£) as  follows:  Set 
— l^t,C0’— ^  =  — »  cll00se  bounded  measurable  V^(t)  z  4  (Pf  (t ,  4  ;F)  t  e  [t^.T]. 
Denote  J?2  (t ,  t^  ;_F] )  =  P^  (*-»**()*— ^*  Having  chosen  bounded  measurable 
V.(t)  e  Vt(P.(t,t0;F)),  t  e  [t0,T],  for  i  -  1,  ...,  k,  let  P^t.^jF)  = 


4  c  e  t c q > T J •  Using  lemma  2.6.3,  for  k  >  1: 

k 


d(4(^b0;F)-Pk+1(t,tn;F)) 

dt 


=  nt,Vk-1(t),Pk(t,t0;F))-I(t,Vk(t),4  (t,t  ;F)) 


i(t,Vk(t),Pk(t,t0;F))-I(t,Vk(t),Pk+J(t,t0;F)) 


A(t,Vk(t))(4(t,t0;F)-Pk+1(t,t0;F))  + 
(Pk(t.t0;F)-Pk+1(t,t0;F))A(t,Vk(t))-  .  (2.6.9) 


1 


Since  P  (t.,cQ;F)  ^>K-fi^rO’ 1  ' 


2.6.9)  implies  that  for  h  >  i: 


0  t  (  [ tQ ,T] 


(2.6.10) 


(2.6.11) 


(2-6.12) 


Therefore,  there  exists  P“  (t ,  ^‘.ii)  such  that 

lira  P^(t,t0;F)  =  P°(t,t0;F) 
k  -® 

Let  us  define,  for  k  >  1,  the  matrix  P*(t,t0;F)  which  satisfies 
£k(  t,t0;F)  =  2.(c,4.i(t),Pk_1(t,t0;F));Pk(t0,t0;F)  «  F 
Clearly,  ?k(t,t0;F)  2  0,  t  [tg.T],  and 


d(P  (-t0^)__4(t-t0,;^  =  A(t,Vl._1(t)(Pk_1(t,t0;F)  Ct,t0;F)) 

+  (4_i(t.t0;F)  -  (c’-k-i(t))  • 

(2.6.13) 

Since  A(t,V,  (t))  is  bounded  measurable  in  [ tQ ,T] ,  taking  limits  on  both 
sides  of  (2.6.13)  and  using  (2.6.11),  (2.6.12)  we  have 

r(c,V°(t),?°(t,tn;F))  =  lim  Pk(t,t0;F)  =  lim  P^ft , tQ;0  =  £  (t,tQ;£) 

—  u  k-*» 

(2.6.14) 

where  V°(t)  t  \JC <P° (t , tQ ;F)  ,  t  €  UQ,T]. 

Note  that  the  choice  of  the  sequence  '^(t)}^  is  nonunique  and  so 

„  X. 

the  sequence  {P.(t,t0;F) }“=1  thus  constructed  is  nonunique.  Let  {V.(t)>.=1 
be  another  chosen  sequence  where  for  i  c  1,  V^t)  e  ht  (£t  (t ,  tQ;F) )  and 
P^^Cq-.F)  =  0.  li^Ct^oJF)  =  P^  (t,tQ;F).  Let 


lim  ^(t.tgjF)  *  £°(t,t0;F) 


(2.6.15) 
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Then  P° (c , C^FO  also  satisfies 

P°(c,c0;F)  -  ^(t,V°(t),P°(c,c0;F))  ;  P°(t0>t0;F)  =  F  (2.6.16) 

where  V°(t)  e  b.(P°(t,t0;F)) ,  t  t  [tg.T]*  Using  lemma  2.6.2,  we  have 

P°(t,t0;F)  -  P°(t,t0;F)  =  y(t ,V°(t) ,P°(t > tQ;F) )  -  ?(t,V°(t) ,P°(t,t0;F)) 

*  I(t,V°(t),P°(t,t0;F))  -  y(t,V°(t),P°(t,t0;F)) 

=  A(t,V°(t))(P°(t,t0;F)  -  P°(t,t0:,F))  + 

(l°(t»c0;F)  -  P°(t,t0;F))A' (t,V°(t))  .  (2.6.17) 

We  conclude  that  P  (t,t^;F)  i  P  (t,tg;F),  t  e  [t^.T].  We  can  interchange 
between  P°(t,t0;F)  and  P°(t , LqCF)  in  (2.6.17)  to  obtain  P°(t,t0;F)  i  P°(t,t0;F) 
Therefore  we  have  the  uniqueness  of  the  function  P^t.tgjF) . 

Let  V(t)  be  an  arbitrary  bounded  measurable  nxm  matrix.  We  have  as 
before : 


P 

-v 


=  I(t,V(t),Pv(t,t0;F))  -  I(t,V0(t),P0(t,tc;F)) 


-  I(t,V(t),Pv(t,t0;F))  -  ?(t,V(t),Po(t,t0;F)) 

“  A(t,V(t))(Pv(t,t0;F)  -  P°(t,t0;F)) 

+  ~  ZO(t,t0;F))A'(t,y(t))  (2.6.17) 

and  so  ^(^tgjJF)  >  _P  c  c  [  ,T  ] .  This  completes  the  proof  of 

the  theorem.  Note  that  the  proof  also  gives  an  explicit  algorithm  to  find 
P°(t,t0;F). 
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Definition  2.6.4:  The  set  ,  . .  _ 

p(t,t0;F)  =  (A(t)-V(t)01(t)"'  ;i)+P(t,t0;r)(A(t)-V(t)D  (t))' 

(t)+<.in-V(t)jD2(t))R(t)  (j^-V^(t)D0(t))  1  ;  P(t0,t  ;F)  -  F 

V(t)(a(c)+D2(t)R(t)Dj(c))  =  P(c,t0;F)Dj(t)+R(t)0'(t)  (2.6.18) 

is  called  the  generalized  Matrix  Riccati  Differential  Equation.  The  unique 
solution  £(t»tQ;£) ,  t  £  [t^jT],  is  called  the  Riccati  function,  which  is 

also  the  minimal  function  with  respect  to  the  solution  sec  RT 

.  C° 

"**  5,-j (c)  >  0.,  then  (2.6.18)  reduces  to  a 

single  nonlinear  matrix  differential  equation: 


?(t,t0;F)  =  (A(t)-R(c)^(t)l"i(c)Di(c))P(c,to;?)-fP(t,c:!;F)(A(t) 
-R(t)D^(t)L.“1(t)Di(t))' 


1(t)D|(t)P(t,t0;F)+R(t)-R(t)D2(t)A_1(t)D^(t)R(t) 

— ^C0,t:0’— ^  =  —  ‘  (2.6.19) 

Equation  (2,6.19)  is  the  Riccati  Differential  Equation. 

In  the  general  case,  for  a  fixed  bounded  measurable  V(t),  t  t  [t  ,1]: 

(2.6.20) 


^v^t,C0’-l^  -  cq;— 2^  if  Li  -  I9 

Let  Xj/t)  ^  ^(.(ICt.tQ;^)),  t  e  [tQ,T],  where  PXt.tgjF^)  is  the  Riccati 
function  satisfying  (2.6.18).  By  theorem  2.6.3,  we  have  for  -  F^; 

-(C,C0;-2)  -  ^v1(c’t0;-2)  -^i(t’to;M)  =  I^»co;Ii}  ;  t  £  [t0,T]  .  (2.6.21) 


Measure  tneoretic  approaches  to  probability  theory  can  be  found  in  Loeve 
[2J,  Doob  [1],  The  notion  of  statistics  as  used  here  was  introduced  by 
Halraas  and  Savage  [13].  The  term  observation  statistic  is  used  so  as  to 
conform  with  physical  interpretation.  Conditional  expectation  and  condi¬ 
tional  distribution  of  a  random  variable  (or  vector)  are  treated  in  detail 
by  Doob  [Ij.  Conditional  independence  of  sub-o-algebra  was  treated  by 
Meyer  [14].  This  is  a  more  general  and  more  intuitive  definition  of  inde¬ 
pendence.  Gaussian  random  vectors  and  Gaussian  random  processes  are  treated 
by  Doob  [1],  Loeve  [2],  Cramer  [15],  Davenport  and  Root  [3].  Gaussian  white 
noise  process  viewed  as  the  formal  derivative  of  a  Weiner  process  is 
treated  by  Wonhair.  [4],  McKean  [16],  ltd  [17];  Gaussian  white  noise  process 
viewed  as  a  generalized  process  can  be  found  in  Tse  [5],  Gel'fand  and 
Vilenkin  [18]. 

Linear  transformation  of  a  Gaussian  Vector  is  treated  by  Davenport  and 
Root  [3],  Cramer  [15],  Doob  [1].  Stochastic  differential  equations  are 
studied  by  ltd  [17],  Stratonovich  [IS],  Wong  and  Zakai  [20],  Tse  [5],  Clark 
[21].  Different  interpretations  to  the  stochastic  differential  equations 
are  possible,  some  are  in  accordance  with  physical  interpretation  [19], 

[21]  while  some  in  terms  of  mathematical  rigor. ^  ^  In  the  linear  case, 
all  different  interpretations  are  equivalent.  The  treatment  used  in 
Section  3  is  consistent  with  all  interpretations.  The  diffusion  process  is 
treated  following  Wonham  [22]  using  Ito's  interpretation,  for  a  detailed 
discussion  on  the  differential  operator  of  a  diffusion  process,  see  Dynkin 

[7] .  The  proof  of  Ito's  integration  formula  is  given  by  ltd  [17],  Skorokhod 

[ 8 ]  ,  Wonham  [ 4 ] . 


Matrix  Riccati  Dif  £ere:._«_  equations  are  not  treated  in  detail  in  ti.e- 
exi.sting  literature.  Deyst  and  Price  [28],  Sorenson  [29]  and  Aoki  [30] 
considered  the  matrix  Riccati  difference  equation  which  appears  in  filtering 
problems.  Their  considerations  are  restricted  to  a  special,  yet  a  large 
class  of  problems.  The  treatment  given  here  is  n  ,  and  the  intrinsic 
properties  of  the  matrix  Riccati  difference  equatiox  are  revealed.  The 
definition  of  generalized  matrix  Riccati  difference  equations  and  Riccati 
sequences  are  due  to  the  author. In  the  continuous  case,  Kleinman  [31], 
Wonham  ( 32 ]  had  made  detailed  instigations.  The  approach  used  here  is  due 
to  Wonham  [32].  The  generalization  given  in  Section  6  is  new,  and  the 
definition  of  generalized  matrix  Riccati  differential  equations  and  Riccati 
functions  are  due  to  the  author.  The  motivation  for  this  generalization  is 
to  bring  out  the  most  intrinsic  properties  of  the  equation  and  its  solution. 
As  we  shall  see  in  later  chapters,  this  generalization  allows  us  to  under¬ 
stand  the  structural  behavior  of  estimators  and  closed  loop  control  systems. 


CHAPTER  III 


OBSERVER  THEORY  FOR  DISCRETE-TIME  LINEAR  SYSTEMS 

3.1  Introduction 

The  problem  of  estimating  the  state  variables  of  a  dynamical  system 
given  observations  of  the  output  variables  is  of  fundamental  importance 
i.n  i_ne  design  of  an  optimal  control  system.  If  one  considers  the  class  of 
linear  systems,  then  there  are  two  approaches  available  in  the  literature. 

If  the  output  variables  can  be  measured  exactly, and  if  there  are  no  other 
stochastic  disturbances  acting  on  the  system,  then  one  can  use  a  determin¬ 
istic  observer  (see  references  [35],  [36]).  On  the  other  hand,  if  all  the 
output  variables  are  corrupted  by  additive  white  noise,  then  one  can  use  a 
Kalman  filter  (see  references  [39],  [40],  [37],  [10])  for  state  estimation. 

The^e  are  many  cases  in  which  some  of  the  output  variables  are  noise- 
free  while  others  are  noisy.  One  can  argue  that  no  measurement  is  exactly 
noise- free.  On  the  other  hand,  there  are  many  engineering  systems  in  which 
the  accuracy  of  measuring  one  variable  is  much  greater  than  the  accuracy 
of  measuring  some  others.  In  such  problems  the  measurement  covariance 
matrix  is  almost  singular  and  it  can  lead  to  ill-conditioned  matrices  and 
numerical  problems.  Thus,  one  can  attempt  to  model  the  very  accurate 
measurements  as  being  deterministic. 

The  main  purpose  of  this  chapter  is  to  examine  this  class  of  problems. 
In  this  contribution  we  examine  the  state  estimation  problem  for  linear 
discrete -time  time-varying  dynamical  systems.  The  continuous  time  case  will 
be  considered  in  Chapter  IV. 
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The  structure  of  this  chapter  is  as  follows.  In  Section  3.2,  we  focus 
our  attention  to  time-varying  deterministic  systems,  we  define  the  notion 
of  a  deterministic  observer  and  estimator,  the  class  of  equivalent  observers, 
and  the  class  of  minimal  order  observers.  In  Section  3.3,  we  extend  the 
deterministic  notion  to  the  class  of  stochastic  systems  where  we  show  that 
the  class  of  equivalent  observers  yield  unbiased  estimates.  Then  we  determine 
the  class  of  observers  that  yield  minimum  variance  estimates  by  formulating 
the  problem  as  finding  the  minimal  sequence  of  a  certain  solution  set  and 
then  make  use  of  theorem  2.5.3;  we  then  prove  that  these  observers  yield 
indeed  the  conditional  mean  estimates  of  the  state.  Naturally,  if  the  obser¬ 
vation  covariance  matrix  is  positive  definite  one  obtains  the  well-known 
Kalman  filter.  In  Section  3.4,  we  examine  in  detail  the  case  that  some 
measurements  are  noisy  while  others  are  noise  free.  Under  these  conditions 
we  show  that  the  order  of  the  minimal  order  observer  is  less  than  that  of 
the  state  to  be  estimated.  In  Section  3.5,  the  notion  of  detectability  is 
defined  and  the  relation  between  detectability  and  observability  of  dis¬ 
crete  linear  system  is  considered;  also  in  this  section,  we  generalize  the 
results  of  Kalman  [41]  on  deadbeat  deterministic  observers  to  the  time 
varying  case.  Using  the  concept  of  detectability,  we  derive  necessary  and 
sufficient  condition  for  the  minimum  error  covariance  to  be  uniformly 
bounded  and  to  have  a  steady  state  behavior.  This  is  carried  out  in  Section 
3-6.  In  Section  3.7,  we  have  general  discussions  on  the  approaches  and  re¬ 
sults.  In  Section  3.8,  detailed  literature  connected  with  the  development 


in  this  chapter  is  listed. 
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3.2  Classes  of  Observers  for  Deterministic  Systems 

In  this  section  we  shall  consider  a  linear  time-varying  discrete  system 
§1  described  by 


(state  eq.)  ,x(k  +  1)  =  A(k)x(k)  +  B(k)u(k) 
(output  eq.)  £(k)  =  £(k)x.(k) 


S 


where  k  =  0,  1,  2,  ....  :c(k)  e  Rn,  u(k)  e.  Rr,  ^r(k)  e  Rm,  and  £(k)  is  of  rank  m 

Let  M  be  the  set  of  all  mxn  matrices  with  real  entries.  If  m  1  n,  the 
mn 

null  space  of  a  matrix  M  t  M  will  be  denoted  by  N(M)  =  (x  e  Rn;  Mx  =  0  t  Rm 

—  mn  —  —  —  -m 

Definition  3.2.1:  Let  C(k)  e  M  be  of  rank  n;  the  set 
- - mn  -  - 

3(C(k);m,s,n)  =  (T(k)  e  M  :N(T(k) )DN(C(k) )  =0  £  Rn} 

—  -  sn  ~  —  — n 

is  called  the  set  of  complimentary  matrices  of  order  s  for  £(k)  if  s  t  n  -  m. 

We  note  that  T(k)  e  3(C(k)  ;ra,s,n)  if  and  only  if  there  exist  P(k)  , 

\Kk)  of  appropriate  dimensions  such  that 


P(k)T(k)  +  V (k)C(k)  =  I  (I  e  M  )  .  (3.2.1) 

—  —  — n  —i n  nn 

Definition  3.2.2:  A  discrete  linear  time  varying  system  of  dimension 
s  >  n  -  m  described  by  the  relation 

G1:  z(k  +  1)  =  F(k)z(k)  +  D(k)y.(k)  +  G(k)u(k)  (3.2.2) 


is  called  an  s-order  observer  for  the  system  if  by  some  appropriate 
choice  of  z(0) ,  we  have 


z(k)  =  T(k)x(k)  ;  for  all  k  =  0,  1,  ...  (3.2.3) 

for  some  T(k)  e  <l(C(k) ;m,s ,n) ,  k  =  0,  1,  2,  ...  .  We  shall  also  say  that 
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the  observer  is  described  t-  '  refer  to  such  an  observer  by  the 

symbol  ©,^. 

If  ©^  is  an  s-crder  observer  for  S^,  than  by  an  appropriate  choice  of 
(0) ,  we  can  reconstruct  x(k)  by 

w(k)  =  P(k)z(k)  +  V(k)y(k)  =  ?(k)T(k)x(k)  +  V(k)C(k)x(k)  =  x(k)  (3.2.4) 


where  P(k) ,  V(k)  are  chosen  to  satisfy  (3.2.1). 

In  the  following  theorem,  we  prove  that  a  class  of  observers  can  be 
constructed  for  any  linear  discrete  L.'me  varying  system. 

Theorem  3-2.3:  Let  T  =  •.T(k)}.  ^  be  any  sequence  of  matrices  in  such 
chat  jT(k)  f  u(C(k) ;m,s,n).  Then,  there  exists  an  s-order  observer,  ©*,  for 


o 

V 


Proof:  The  proof  is  a  constructive  one  in  which  an  explicit  form  of  ©^  is 
obtained.  Let  T(k;  c  iHC(k)  ;m,s,n) ,  k  =  0,  1,  ...,  be  given.  Pick 


F(k)  =  T(k  +  l)A(k)P(k) 

(3.2.5) 

D(k)  =  T(k  +  l)A(k)V(k) 

(3.2.6) 

G(k)  =  T(k  +  l)B(k) 

(3.2.7) 

(3.2.1),  k  =  0,  1,  2 .  Then 

z(k  +  1)  -  T(k  +  i)x(k  +  1)  =  T(k  +  l)A(k)P  (k)  (z.(k)  -  T(k)x(k))  .  (3.2.8) 

Therefore,  if  we  choose  z(0)  =  T(0).x(0) ,  we  obtain 

z(k)  =  T(k)x(k)  ;  k  =  0,  1,  2,  ...  .  (3.2.9) 


The  observer  described  by  the  given  sequence  T  has  the  explicit  form: 


Q~:_z(k-rl)  =  T(k+l)A(k)P(k)z(k)  +  T(k+l)A(k)V(k)y.(k)  +  T(k+l)B(k)u(k)  . 

(3.2.10) 

To  an  observer  we  associate  an  estimator  described  by  (see  Figure 


z(k+l)  =  T(k+l)A(k)P(k)z(k)  +  T(k+l)A(k)V(k)x(k)  +  T(k+l)B(k)u(k) 
w(k)  =  J?(k)^(k)  +  V(k)^(k) 


(3.2.11) 


where  _P  (k) ,  V/k)  satisfy  (3.2.1)  for  the  fixed  T(k) ,  k  =  0,  1 .  By 

setting  ^(0)  =  T (0)x(0) ,  w(k)  will  equal  2<(k)  by  (3.2.4).  But  in  most 
cases,  the  initial  state  j<(0)  is  unknown.  We  shall  fix  the  initial  condi¬ 
tion  for  the  observer  by  the  rel'  ion 

z(0)  =  T(0)a  (3.2.12) 

where  the  vector  a  is  a  guess  for  3£(0) .  Thus  a  is  any  vector  in  Rn,  and 
the  possible  values  of  z,(0)  will  be  in  the  range  space  of  T(0). 

i  00 

Let  V  i  [ V (k) }.  be  any  sequence  of  matrices  in  M  .  Let  us  associate 

k=0  nm 

with  the  given  sequence  a  sequence  of  sets  where 

3  ,  .  =  { T(k)cM  I P(k)T(k)+V(k)C(k)  =  I  for  some  P(k)sM  ;s^n-m}  , 

V(k)  —  sn  —  ~  ~  —  — n  —  ns 

l-  =  r  i 

lx  v  j  X  j  •  •  •  • 

CO 

If  T  =  { T ( k ) } .  n  is  a  sequence  of  matrices  in  M  such  that 
—  k=u  sn 

l.(k)  c  k  =  0,  1,  2,  then  we  shall  in  short  write  T  e  3^.  Now 

let  T  e  ?  ;  by  theorem  3.2.3  we  can  associate  every  such  T  with  an 


-49- 


observer  and  an  estimator  Thus,  we  can  associate  with  any  fixed 

V  a  class  of  observers  of  different  orders. 

Suppose  that  for  a  fixed  V,  the  sequence  of  matrices  { (I  -  V(k)C(k))}, 

k  =  0,  1,  2,  has  rank  n  -  p;  then  the  class  11^  =  {©^jT  e  3^,  and 

T(k)  e  M  /  ,  has  full  rank,  k  =  0,  1,  ...}  is  called  the  class  of  minimal 

-  n(n-p)  - 

order  observers  associated  with  V. 

Definition  3.2.4:  Let  be  a  linear  discrete  system  described  by 

x^k  +  1)  =  F-^CkJXj^k)  +  Gx  (k)u  (k) 

L  :  (3.2.13) 

^(k)  =  C^Wx^k)  +  D1(k)u(k) 

with  e  CRn.  We  shall  say  that  L^,  described  by 

x2(k  +  1)  =  £2(k)x2(k)  +  G2(k)u(k) 

L2:  (3.2.14) 

i2(k)  =  C2(k)x2(k)  +D2(k)u(k) 

with  XoCO)  t  X2  C  Rn,  is  an  equivalent  representation  of  if  for  any 
6^  e  there  exists  a  X2  such  that 

^l^’-l’-k^  =  ^-2^k;-2’-k^  V  ^k  =  (u(i)  (3.2.15) 

where  ^(lt;3^,u^)  is  the  output  of  the  system  for  (0)  =  6^  and  applied 
control  u,  . 

Equivalent  representations  may  not  be  symmetric,  i.e.,  if  L is  an 
equivalent  representation  of  L1  this  does  not  imply  that  L.  is  an  equivalent 
representation  of  L2>  If  and  L2  are  both  equivalent  representations  of 
each  other,  then  we  say  that  and  L?  are  equivalent . 


1 


£e  shall  sav  ttn-it  tv..: 


,1  ......  *1  ,  ^  I 

- . ,  f_,,  art*  ccaiva»«.-t  j  :  <  _  end  t  . 


aye  equivalent-  Let  -'  S  -  ~.vrdcr  cisc-rver;  by  sene  I  Inear  transform- 

tien  on  cae  state,  ve  ca  *. .a -  .  I  see  a  at  *<?  c construct  as  s*— oraer  — 

a  * 

server  ££,  Is  ax  equivalent  representat  a  an.  ii  £‘.  i>.k  s*  _  -  - 

f  i 

Interestingly  earaga,  If  ve  restrict  ate  possible  initial  ..auditions  ni  the 


observer  stats,  then  ter  any  s-order  observer,  S_,  ve  can  find  an  s’-order 

ft. 

observer,  vhich  is  an  eauivalenc  representation  of  0_,  vith  s  is. 

»  31 

nheorem  3.2.5:  Let  V  =  V(k)-^  be  a  fixed  sequence  of  matrices  »a  H  , 
che  class  of  observers  ©Z.  3"  -a  I..,  are  equivalent. 

ft  4 

Proof:  Let  £*,  St  be  any  tvo  estimators,  T,  T  s  described  by 

"  ft  i  » 

z(fc4-l)  =  T(k4-I)A(k)?{k)z(k)  -5-  T(kv!>A(k)y(k)v(k)  -r  T(k4*l)B(k)u(k) 

€*:  (3.2 

«(k)  =  ?(k)z(k)  +  V(k)v(k)  :  z(0)  £  S  »  ^(QKjL  :  ^ 


(3.2.16' 


z(krl)  =  T(k4*l)A(k)?(k)z(k)  4-  T(k4-i)A(k)Y(k)v(k)  4-  T(k4-l)B(k)u(k) 


v(k)  =  P(k)z(k)  4-  V(k)v(k)  ;  z(0)  -a  S  =  (T(O)aii  e  R  } 


(3.2.17) 


Let  £(0)  =  T(0)a_^  a  S  be  the  initial  condition  for  G*.  Choose 
z(0)  =  T(0)a^  c  S  be  the  initial  condition  for  g1,  then  for  all  v(0) : 

w(0)  =  P (0)1(0)^  4-  V(0)x(0)  =  P (0)1(0)^  4-  V (0)^(0)  =  v(0)  .  (3.2.18) 


Assume  that  w(k  ~  1)  =  w(k  -  1),  then 


w(k)  =  P(k)T(k)A(k)w(k  -  1)  4-  V(k)^(k)  +  P(k)T(k)B(k)u(k) 


=  P(k)T(k)A(k)w(k  -  1)  4-  V(k)x(k)  4-  P(k)T (k)B(k)u(k) 


=  w(k) 
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?rea  (3.2.18)  and  (3.2  l* '  we  conclude  that 

wft)  =  w(k)  fcr  all  k  .  (3.2.20) 

Conversely  if  z^(O)  =  1(0)^  e  S ,  pick  2/ CO  =  T(0)^7  c  S'  then  we  also  have 

that  the  output  of  and  that  of  si  are  the  sene-  Thus  ®|  are  equiva- 

11  - 
lent,  and  G*  are  equivalent. 

A  & 

Thus,  for  a  fixed  sequence  V,  we  can  associate  with  a  class  of  equiva¬ 
lent  observers  G\lr  T  »;  Khea  V  ranges  over  all  possible  sequences,  we 

obtain  different  classes  of  observers  parameterized  by  the  sequence  V.  In 
a  vague  sense,  the  class  of  observers  T  e  5^,  utilise  the  sane  amount 
of  incoming  information  provided  by  the  observations  v(k)s  k  =  0,  1,  ...  . 
The  notion  of  efficiency  of  a  system,  as  regard  to  the  processing  of  in- 
coning  information,  is((n  a  loose  sense) a  ratio  between  information  utilized 
and  the  complexity  of  the  system.  Thus  for  a  fixed  V,  the  most  efficient 
system  associated  with  it  is  the  class  of  minimal  order  observers,  .  In 
view  of  the  above  discussion,  the  design  ot  appropriate  observer  for  esti¬ 
mation  and  control  purposes  can  be  split  into  two  distinct  steps:  1)  to 
rind  the  appropriate  V  which  specifies  the  operating  performance  of  the 
class  of  observers,  2)  to  find  an  observer  in  the  class  of  minimal  order 
observers,  . 

3 • 3  Optimum  Classes  of  Observers  for  Linear  Stochastic  Systems 
Let  us  consider  the  stochastic  system  described  by 

x(k  +  1)  =  A(k)x(k)  +  B(k)u(k'i  f  £(k) 


£(k)  =  C(k)x(k)  +  n(k) 


(3.3.1) 


where  x(0),  ^(0),  ^_(G),  ^{i  » , _  art-  independent  Gaussian  rondo-— 

vectors  with  sracistical  lav . 


x(0) 

G  <2L  *  — 

;  “0  ^ 

(3.3.1) 

i(k) 

G(0,  R(k)) 

;  R(k)  >  0 

(3.3.3) 

-(k) 

-  G  <0,  £(fc)) 

^(k)  >  0 

(3.3.4) 

The  control  u(k),  k  =  0,  1,  _ ,  is  an  arbitrary  but  known  sequence. 

Let  V  =  ‘  V(k) ;*  .  be  an  arbitrary  sequence  of  natrices  in  H  -  If  we 
—  k=0  nc 

usa  an  estimator  6^,  T  z  3  for  S,  to  generate  an  estimate  of  x(k) ,  then 
1  V  z 

the  error  e^(k)  4  v(k)  -  x(k),  can  be  computed  f roc  (3.2.11)  and  (3.3.1). 

3y  picking  z(0)  *  T(0)x  ,  the  error  dynacics  are  given  by 

—  _  o 

e(k-fl)  =  [I  -V(k+l)C(k+l)]A(k)e(k)+V(k+l)n(k+l)+(V(k+l)C(k+l)-I  )C(k) 

—  — q  —  —  —  —  —  n 

(3.3. 

e(0)  =  [1  -V(0)C(0) ] [x  -x(0)]+V(0)n(0) 

—  — n  —  —  — o  —  —  — 

So  explicitly  (3.3.5)  reveals  that  all  estimators  T  z  3^,  give  the  same 
error  dynamics  ,which  in  some  sense  reflect  the  state  of  uncertainty  of  the 
system  From  (3.3.5)  we  see  that 

E[e(k) ]  =0  ;  k  =  0,  1,  2,  ...  .  (3.3.6) 

Therefore »associated  with  an  arbitrary  V,  we  have  a  class  of  equivalent 

observers  whose  associated  estimators  yield  unbiased  estimates.  Our  aim 

* 

now  is  to  find  the  optimum  V  which  will  result  in  minimum  error  covariance. 
From  (3.3.2)  to  (3.3.6)  we  see  that  the  error  covariance  will  propagate 


according  to  the  matrix  difference  equation: 
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:(k-rl)  =  [I  -V(k-l)C(k-I)  ]  [A(k) I (k)A'  (k)-rR(k)  J  [1  -V(k+l)C(fcrl>]  * 
—  — n  —  —  —  - n  —  — 

^V{k^l)0(k>l)V'  (fc+l)  ;  &  -•  G,  1,  ... 

KO)  =  fi  -v(0)c(0)]z  (i  -vco)c(o)]  ’^'(0:^(0)^  (0) 

—  a  -  —o  — n  — 


(3.3.7) 


where 


I(k)  =  E{e(k)e'(k>}  .  k  -  0,  1,  ...  .  (3.3.8) 


Defining 

A(— 1)  =  I  ;  R(-l)  =0  (3.3.9) 

—  -n  —  — 

equation  (3.3.7)  can  be  written 

Z(k-i-l)  =  [I  -y(k+l)C(k+l) ] [A(k)E (k)A' (k)+R(k) ] [I  -V(k+l)C(k+l) ] ' 

—  — q  —  —  —  —  —  —  n  *•*  — 

+V(k+l)£(k+i) V* (k+1) ;  k  =  -1,  0,  1,  ... 

(3.3.10) 

I(-l)  =  Z 

—  — o 

When  V  ranges  over  all  possible  nxm  matrices,  we  generate  a  solution 

set  &  ^  of  (3.3.10).  For  the  optimum  estimation,  we  would  like  to  find  a 

* 

sequence  V  which  will  give  rise  to  the  minimal  sequence  with  respect  to 
the  solution  set  fi_^.  iy  comparing  (3.3.10)  with  (2.5.1),  we  have  the 
following : 

k  oo 

Theorem  3.3.1:  A  unique  minimal  sequence  {£  (k)}^_Q  with  respect  to  the 
solution  set  $  of  (3.3.9)  exists  and  is  given  by 

Z*(k  +  1)  =  A*(k)  -  V*(k  +  l)C(k  +  l)A*(k)  ;  k  =  0,  1,  ... 

l\o)  «  E  -  E  C'(0)[C(0)E  C’(0)  +  $(0)  ]_1C(0) Z 


(3.3.11) 


where 


i*(k)  =  A{k)£*Ck)..'i>#  -  I.k; 


k  0,  1,  ...  (3.3.12) 


and  V*  (k>  £  -  V  •_  AlCik)-. "  (k-I)C’  (k)-H£(k)  ]  =  A  (k-l)C  ’  (k) } , 

k  =  1,  2,  ...  . 

If  either  0(k)  >  0,  k  =  0,  1,  ....  or  C(k~l)K(k)C* (k-H)  •  0, 
k  =  0,  1,  —  (or  both),  then  the  unlove  Riccati  sequence  is  given  by 


(k-rl)  =  2“ (k)-l" (k)C’  (k+i)  [C(k-rl)l‘  (k)C'  Qc*l)4£(k*-1)  j'^CCk-rl)!  (k^ 


(3.3.13) 


-i 


2  <o)  =  z  -z  c’(0)fc(0)z  c'(o)-5$(o)]  -c(0): 


k  «  0,  1, 


where  L_  (k)  is  given  by  (3.3.12)  „  and  the  unique  ;\T  (k)  ‘j._q  which  gives 
rise  to  the  Riccati  minimal  sequence  is  given  by 

V*  (k)  =  A*(k  -  1)C' (k) [C(k)£*(k  -  l)C’(k)  +5>(k)]_1  ;  k  =  0,  1,  ...  .  (3.3.14) 


The  proof  of  this  theorem  follows  from  theorem  2.5.3  directly  by 
identifying 

C(k  +  1)  -*  D(k)  ,  R(k)  -  ^(k)  =  £2(k)  (3.3.15) 

E(k>  -  P^k.-l;^)  ,  £(k)  -  R(k)  .  (3.3.16) 

Theorem  3.3.1  implies  that  an  optimum  class  of  observers  is  specified  by 

any  sequence  {V  (k)^_Q  where  V  (k)  e  (I  (k  -  1))  inductively, 
k  =  0,  1,  ...  with  l_  (k)  given  by  (3.3.11),  and  (3.3.12),  k  =  -1,  0,  1,  ...  . 

In  the  special  case  when  £(k)  >  £,  or  C(k  +  l)R(k)C 1 (k  +1)  >0,  k  =  0,  1,  ...» 

"ft  CO 

then  there  is  a  unique  class  of  optimum  observers  specified  by  {V  (k)}k„Q> 

given  by  (3.3.11).  In  fact,  one  can  show  that  an  observer  with  an  initial 


condition  z(0)  =  1(0)3^  is  in  some  sense  equivalent  to  the  concept  of  un¬ 
biased  linear  estimator  {see  Section  3.7),  and  thus  an  optimum  class  of 
observers  is  also  an  optima  class  of  linear  unbiased  estimators.  In  the 
rest  of  this  section,  ue  shall  show  that  when  u(k)  is  known ,  the  estimator 
generated  via  an  observer  T  e  and  ics  associated  estimator 

is  the  conditional  mean  of  x(k) .  This  reflects  the  truly  optimum  nature 
of  the  optimum  classes  of  observers. 

Since  u(k) ,  k  =  0,  1,  ...,  are  known  a  priori,  we  may  assume  them  to 
be  zero  without  loosing  generality.  Now  consider  §2  with  control  sequence 
equal  to  zero.  By  the  Gaussian  assumption,  the  conditional  expectation  of 
x(k) ,  denoted  by 

x(k|k)  =  E(x(k)|F(k)}  ;  F(k)  k  Ffc(i),  i  =  0,  1,  . . . ,  k)  (3.3.17) 

equals  almost  surely  to  some  linear  functional  of  {y.(0) ,  ...,  x(k)}.^ 

00 

Lemma  3.3.2:  (Weiner-Hopf  Equation)  Let  {w(k)}^_Q  be  a  sequence  of  random 
vectors  such  that  w(k)  is  a  linear  functional  of  ^(0) ,  . . . ,  x00  *  If  in 
addition,  w(k)  satisfies  for  k  =  0,  1,  ... 

E[w(k)x'(i)]  =  E[x(k)x'(i))  i  =  0,  1,  ...,  k  (3. 3. IS) 

then  w(k)  =  x(k|k)  a.s.  for  all  k. 

The  proof  is  given  in  the  Appendix  B.  An  immediate  consequence  of  this 
lemma  is  the  Projection  Theorem. 


TThe  superscript  is  used  to  indicate  that  the  stochastic  system  §2  is  being 
considered. 
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Theorem  3.3.3:  (Projection  '  •  r%  ,  U<  w(k)?*_0  be  a  sequence  of  random 
vectors  such  that  w(k)  is  a  I  ■.  sr  iur..  t ’Vnal  of  {^(0) :  — ,  jKk)}.  Let 
e(k)  ^  w(k)  -  x(k),  k  =  0,  1.  .  .  •  if  e(k)  satisfies,  for  all  k, 

Ele(k)v‘(i)i  :  ;  i  =  0,  1,  ....  k  (3.3-19) 

then  v(k)  =  j<(k*k)  almost  surely  for  all  k.  [V?e  shall  refer  to  Equation 

(3.3.19)  as  the  Projection  equation.] 

For  any  fixed  sequence  V  =  { V^(k) ) j__q »  £ke  output  of  the  estimator 
T  s  ov,  at  time  k  is  clearly  a  linear  functional  of  {^(0) ,  ...,  v(k)}. 

In  the  fo llovmg,  we  shall  prove  that 

Etei‘(k)x'(i)}  =  0  ;  i  =  0,  1,  ...,  k  (3.3.20) 

*  2 
where  e^  (k)  the  error  of  estimates  if  we  adopt  S  “,  T  e  T~sv,  as  an  estimating 

device,  and  e:  (k)  is  given  by  (see  3.3.5) 

e*(k+l)  =  [1^  -  V*(k+l)C(k+l))[A(k)e*(k)  -  C.(k)  ]  +  V*(k+l)n (k+1) 

(3.3.21) 

e*(0)  =  [1  -  V* (0)C (0) ] (x  -  x(0))  +  V*(0)n(0) 

_  — n  -  —  •— o  —  _  _ 

^  ^  >V 

and  V  (k)  e  (Z  (k  -  1))  inductively,  k  =  0,  1,  ...,  with  £  (k)  given  by 

(3.3.11),  (3.3.12).  Let  us  first  establish  a  lemma  and  a  corollary  which 

will  be  useful  in  later  discussions. 

Lerrma  3.3.4:  Let  (jKk)  be  a  sequence  of  random  vectors  satisfying 

(3.3.20) .  Let  {x(k)}^_Q  be  given  by  (3.3.1)  with  u(k)  =0,  k  =  0,  1,  ..., 
then  for  all  k  =  0,  1,  ... 

E{j[  (k)  1  (k) }  =  -  Z  (k) 

where  _Z  (k)  is  given  by  (3.3.11),  (3.3  12). 


(3.3.22) 
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Proof:  We  shall  use  induction  on  k.  For  k  =  0,  using  (3.3.21),  (3.3.1), 

(3.3.11)  and  the  given  statistical  law: 


E(e*(0)x' (0) ]  =  (I  -  V*(0)C(0)){x  x*  -  E[x(0)x' (0) ] } 

o  o  —  — 


(3.3.23) 


=  -  Z  +  V*(0)C(0}Z  =  -  Z*(0) 

— o  —  —  — o  — 


Assume  that  (3.3.22)  is  true  for  k  =  0,  1,  . ..,  u.  Using  (3.3.21),  (3.3.1), 

(3.3.11),  (3.3.12)  and  the  given  statistical  law,  we  have: 


E[e  (n+l)vc' (n+1)  ]  =  [I^-V  (n+l)C(n+l)  ]  (A(n)E{e  (n)x/  (n)  }A'  (n)-R(n)  ] 


(3.3.24) 


=  -[A*(n)-V*(n+l)C(n+l)A*(n) ]  =  -Z*(n+1) 


The  lemma  is  proved  by  induction. 

oo 

Corollary  3.3.5:  Let  {e^(k)  be  a  sea.uence  of  random  vectors  satisfying 
(3.3.21)  where  _V*  (k)  e  (jf  (k  -  1))  with  (k  -  1)  given  by  (3.3.11)  and 

(3.3.12).  Let  {^(k)}^_Q  be  given  by  (3.3.1)  with  ^(k)  =  £,  k  =  0,  1,  ...  . 
Then  for  all  k  =  0,  1,  ... 

E{e*(k)x'(k)}  -  0  .  (3.3.25) 


Proof:  We  shall  use  induction  on  k.  For  k  =  0,  since  using  (3.3.21), 

(3.3.1),  (3.3.22)  and  the  given  statistical  law,  we  have 

H{eX(0)x'(0)}  =  E{e*(0)x,(0)}C'(0)+E{e*(0)n,(0)} 

(3.3.26) 

=  -I0£'  (0)+v"(0)C(C)^C'  (0)+V*  (0)<£(0)  =0 

Assume  that  (3.3.25)  is  true  for  k  =  0,  1,  ...,  n.  Since  V  (n+1)  e  b  (E*  (n)), 
using  lemma  3.3.4,  (3.3.11)  and  (3.3.1),  we  have 
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E- e  (n+1)^'  (n+1) 


E{je  (n+1  x1  (n+1)  C  ’  (n+1  )+E{ji  (n+l)^' (n+1)  } 
—Z  (n+l)j  '  (n+J  )+V  fr.+l)Q(n+l) 


=  -Wn+l'C'  (n+D+i^Oi+DC'  (n+1 ) - V* (n+1 ) C (n+1  )  E* (n)C’  (n+1) 
=  -f(n+l)C'  (n+l)+IX(n+l)C'  (n+1)  =  0  (3.3.27) 


the  corollary  is  proved  by  induction. 

We  are  now  in  a  position  to  prove  (3.3.20).  The  results  are  stated  as 
a  theorem. 

Tneorem  3.3.6:  Let  {e  (k) }  be  given  by  (3.3.21),  where  V  (k)  e  V  (Z  (k-1)) 

K.  o  K.*“  X 

with  Z  (k-1)  given  by  (3.3.11)  ai.o  (3.3.12).  Let  /v.'  (k) )|._q  be  given  by 

: k 

(3.3.1)  with  u(k)  =  (),  k  =  0,  1,  ...  .  Then  for  all  k  =  0,  1,  .  . . ,  e  (k) 
satisfies  the  projection  equation;  i.e., 

E{g*(k)£' (i)}  =0  i  =  0,  1,  ...,  k  .  (3.3.28) 

Proof:  We  shall  use  induction  on  k.  By  Corollary  3.3.5,  (3.3.27)  is  true 
when  k  =  0. 

Assume  that  (3.3.27)  is  true  when  k  =  0,  1,  ...,  n.  For  i  =  0,  1,...,  n, 
we  have  from  (3.3.21)  and  the  induction  hypothesis  that 

E{e*(n+l)i'(i)}  -  (^  -  V*(n+l)C(n+l)A(n)E{e*(n)Z'(i)}  =0  .  (3.3.29) 

For  i  =  n  +  1,  Corollary  3.3.7  gives 

E{e*(n  +  l)i' (n  +  1)}  =  0  .  (3.3.30) 

Combining  (3.3.29)  and  (3.3.30),  we  have  that  (3.3.28)  is  true  ;and  the 
theorem  is  proved  by  induction. 
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In  view  or  the  Projection  Theorem,  this  means  that  the  estimate  w  (k) 

2 

generated  via  T  is  the  a.s.  conditional  mean  of  Hk)  ,k=0,  . . .  . 

The  results  are  also  true  when  u(k),  !c  =  0,  1,  . . . ,  is  a  nonzero  but  known 
control  sequence,  for  one  car*,  always  subtract  off  the  deterministic  contri¬ 
bution  due  to  the  nonzero  known  control  sequence.  The  general  situation 
where  the  ji(k),  k  =  0,  1,  are  generated  by  feedback  of  the  observation 

sequence,  shall  be  considered  in  detail  in  Chapter  5. 

From  che  above  discussions,  we  note  that  in  the  general  cage,  there  is 
more  than  one  optimum  classes  of  observers  which  will  yield  the  same  per¬ 
formance;  only  in  the  sp  icial  case  when  £.(k)  >  0  or  £(k+l)R(k)^' (k+1)  >  0, 
k  =  0,  1,  ...  (or  both),  there  exists  a  unique  ontimum  class  of  observers. 

3 . 4  Minimal  Order  Optimum  Observers  for  Stochastic  Sy«. terns 

Let  V  (k)  e  ^^(E  (k  “  *))>  k  =  0,  1,  2,  ...,  specify  an  optimum 
class  of  observers.  The  class  of  minimum  order  op-imum  observers  associated 

f  ~  CO  P 

with  t_V  (k) )jc_q  is  where  p  is  the  minimal  order  (or  dimension).  To 

find  the  number  p  amounts  to  finding  the  rank  of  the  matrix  [I  -  V  (k)C(k)]. 

n  — 

We  shall  see  that,  depending  on  the  observation  noise  » e  have  that  the 
minimal  order  optimum  observers  will  have  order  which  ranges  from  n  -  m  to  n. 
Let  us  assume  that  the  observations  are  partly  deterministic,  i.e., 


■cl(k)“ 

n_l  (k) 

X(k)  = 

•  ♦  ♦ 

= 

:x(k)  + 

•  •  . 

y.2  <k> 

c2(k) 

0 

(3.4.1) 


m2 

where  x^(k)  e  R  >  i-2^)  e  R  •  The  vector  ^(k)  ^  -'-e  noise-free  component 
(Figure  3.2).  This  assumption  has  no  loss  of  generality,  fro^by  appropriate 
transformation  of  the  observation  vector,  all  problems  where  the  observation 
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noise  is  degenerate  can  be  put  into  this  form.  We  shall  assume  that 


r^OO 


has  covariance  matrix  ^.{k) , 


V*°  £ 

3(k)  =  ...  :  ... 

0  0 

-ta2m2 


;  fi.(k)>0  5  (k)  c  M  m  .  (3.4.2) 

i  —  1  m^m^ 


Definition  3.4.1:  A  system  with  output  w(k)  is  called  compatible  with 
respect  to  the  noise-free  observation  v„  if  for  k  =  0,  1,  ... 


C9(k)w(k)  =£9(k;-vk)  =  £2(k)  a.s. 


(3.4.3) 


We  shall  also  say  that  an  observer  T  e o is  compatible  with  re¬ 
spect  to  v^if  ics  associated  estimator  T  e  3^,  is  compatible  wita  respect 
co  X2* 

Theorem  3.4.2:  Let  V_  (k)  c  l^_^(Si  (k  -  1)),  with  _E  (k  -  1)  given  by  (3.3.11) 
and  (3.3.12);  any  T  £  is  compatible  with  respect  tc>  the  noise-free 

observation  y2 • 

Proof :  Using  Corollary  3.3.5,  we  have 


V  (k)£(k)  =  E  (k)C* (k)  ;  k  =  0,  la 


(3.4.4) 


Partition  the  matrix  V  (k)  into 


I*(k)  =  [V?(k)  :  V*(k)]  ;  V*(k)  s  M  ,  V*(k)  e  M 

i.  •  nm^  - 2  nm. 


k  =  0,  1,  ... 


(3.4.5) 
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Equation  (3.4.4)  implies 

vj(k)£,(k)  =  Z\  :  k  =  0,  1,  ...  (3.4.6) 

0  m  =  2  (k).  .,)  ;  k  =  0,  1,  ...  (3.4.0 

-  na1 

A. 

where  Z_  (k)  satisfies  (3.3.11)  and  (3.3.12).  The  theorem  follows  from 
(3.4.7). 

In  the  following,  we  shall  consider  the  special  case  where  the  sequence 
of  matrices  {R(k)}^_ are  all  positive  definite.  The  general  case  can  be 

treated  in  a  similar  approach.  Since  by  assumption  j*(k)  >  0,  k  =  0,  1,  .  „ . , 

*  * 

Z  (k)  is  given  by  equation  (3.3.13),  and  V  (k)  given  by  (3.3.14)  specifies 

the  unique  optimum  class  of  observers. 

Lemma  3.4.2:  Let  R(k)  '  0,  k  *  0,  1,  . . .  .  If  the  noise-frea  observation 

* 

v7(k)  R  k  =  0,  1,  ...,  tnen  7  (k)  given  by  (3.3.13)  is  of  rank  n  - 
k  —  0 ,  Jl  |  ...  . 

Proof :  By  compatibility  (3.4.7)  we  have 

rank  Z_  (k)  1  n  -  k  =  0,  1,  ...  .  (3.4.8) 

From  (3.4.6)  and  (3.3.14),  we  deduce  that 

c1(k):*(k)  =  ^(kyvj’tt) 

=  0X (k)i*(k)C1 (k) [L*(k-l)-4* (k-l)c; (k) 

(C2 (k)L*  (k-l)C^  (k)  )_1C2 (k)A* (k-1)  ] 


(3.4.9) 
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where 

A*(k)  =  (k)+C^ (k) [A* (k-l)-A* (k-l)C' (k) 

(C2 (k) A* (k-l)C’  (k) )_1c2 (i-.)A* (k-l)  ]C ’  (k) }_1  >  0  ; 

k  =  0,  1,  ...  .  (3.4.10) 

(In  deriving  (3.4.9)  from  (3.3.14),  a  fair  amount  of  matrix  algebra  is 

k 

needed,)  Let  us  define  the  matrix  F  (k  -  1)  by 

L  (k-l)  k  A  (k-l)Cj (k) (C? (k)A* (k-l)£2 (k) )-1C2 (k)  A* (k-l)  ; 

k  =  0,  1,  ...  .  (3.4.11) 


Now  equation  (3.4.9)  can  be  written  as 

£1(k)A*(k)  =  £1(k)A*(k)C1(k)(A*(k-l)-r*(k-l)]  ;  k  =  0,  1,  ...  .  (3.4.12) 

We  note  from  (3.4.11)  that  if  a  vector  v  e  N(T  (k  -  1))  then  it  must  be  true 

—  —  9 

that  A  (k  -  1)^  e  N(C?(k)).  Now  suppose  that  the  same  vector  v  z  N(Z  (k)); 
then,  from  (3.4.9),  we  conclude  that 

£1(k)A>'(k)C1(k)A>'(k-l)v  =  o  =*A*(k-l)v  e  N(C^  (k)) 


k  =  0,  1,  ...  .  (3.4.13) 


Therefore, 

A*(k  -  1) v  e  N (C (k) )  ;  k  =  0,  1 .  (3.4.14) 

But  from  (3.3.11),  we  have 

0  =  A* (k  -  l)v  ;  k  =  0,  1,  .. . 


(3.4.15) 
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Si  nee  1  (k  -  1)  '0  vk,  ue  r  ave  v  =  0.  Thus, 

>!(£  (k  -  1))H  N C J  (k) )  =  {0>  k  -  0,  1,  ...  .  (3.4.16) 

Clearly,  che  rank  of  £  (k  -  1)  is  n^,  and  so  N(£  (k  -  1))  has  dimension 
n  -  ra9  (3.4.16)  implies  Chat 

rank  (E  (k))  >  n  -  ra?  k  =  0,  1,  ...  .  (3.4.17) 

Equations  (3.4.8)  and  (3.4.17)  imply  that  rank  2“ (k)  =  n  -  m2 . 

Theorem  3.4.3:  Let  R(k)  >  0,  k  -  0,  1,  ...  .  If  the  noise-free  observation 
v^Ok)  s  R  k  =  0,  1,  ...,  then  the  class  of  minimal  order  observers  is 
of  order  n  -  m„ . 

Proof :  From  the  remark  made  at  the  beginning  of  this  section,  one  needs 

* 

only  to  prove  that  the  matrix  [1^  -  V  (k)jC(k)  ]  has  rank  n  -  m7,  k  =  0,  1 . 

(3.3.13)  and  (3.3.14)  give  us 

[I  -  V*(k)C(k)]4*(k  -  1)  =  !*(k)  ,  k  =  0,  1,  ...  .  (3.4.18) 

— n  —  ~  —  — 

it 

3y  assumption,  R(k)  0,  k  =  0,  1,  ...;  thus  4  (k)  has  full  rank  for  all  k. 

* 

By  lemma  3.4.2,  Z  (k)  has  rank  n  -  n^,  k  =  0,  1,  ...;  therefore, 

[1^  -  V  (k)C (k) ]  has  rank  n  -  n^. 

To  end  this  section,  we  shall  give  one  explicit  minimal  order  optimum 
observer  and  its  associated  estimator  for  each  case: 

Case  1:  m2  =  0 

The  class  of  minimal  order  optimum  observer  is  of  order  n,  and  one 
explicit  optimum  estimator  of  minimal  order  can  be  constructed: 

C:  w*(k+l)  =  (I  -V* (k+l)C (k+1) ) A(k)w* (k)+V* (k+l)x(k+l)+J3  (k)u.  (k)  (3.4.19) 


-65- 


where  (V^  (k))^_Q  is  given  by  (3.3.11)  to  (3.3.13).  Jje  _nQti.ce  .that  .this 
is  the  Kalman  estimator  f39l.  (Figure  3.3.) 

Case  2:  m^  =  0 

The  class  of  minimal  order  optimum  observer  is  of  order  n-m,  i.e., 

there  must  exist  P(k)  c  M  ,  .  and  T(k)  z  M.  .  ,  k  =  0*  1,  ...  such  that 

—  n(n-m)  —  (n-m)n’ 


[P(k)  :  V  (k)] 


T(k) 
•  •  • 
C(k) 


=  I 
— n 


(3.4.20) 


Since  T(k)  z  M.  .  and  C(k)  z  M  ,  (3.4.20)  also  implies  that 
~  (.n-m;  n  —  mn 


T (k)V  (k)  =  0  ,  ,  ;  T(k)P(k)  =  I  ;  C(k)P(k)  =  0  .  .  .  (3.4.21) 

—  —  -m(n-m)  —  —  -n-m  —  —  -m(n-m) 


To  specify  one  explicit  minimal  order  optimum  observer,  let  {P  (kH^.Q  be 
any  sequence  of  matrices  such  that 


COOP  ft)  =  0^, 


k  =  0,  i ,  . . 


(3.4.22) 


Let  {T  (k)},_.  be  the  solution  of 
ic*-*  u 


T  (k)V  (k)  =0.  .  ;  T  (k)P  (k)  =  I  ;  k  =  0,  3,  ...  .  (3.4.23) 

—  -  —  (.n-mjm  —  —  -n-m 


The  solution  for  (3.4.23)  exists  and  is  unique  because  we  know  a  priori  that 
(3.4.21)  must  have  solutions  (nonunique).  The  choice  of  fP*(h)^-o 
is  nonunique,  and  is  usually  chosen  so  as  to  simplify  computation. 


’Note  that  the  condition  £(k)V  (k)  =  I  is  automatically  implied  by 
compatibility. 
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Case  3:  0  <  <  m 

The  class  of  minimal  order  optimum  observer  is  of  order  n  -  m,,  i.e., 

there  must  exist  P(k)  c  M  ,  ,  and  T(k)  e  M,  .  ,  k  -  0,  1,  ...  such 

-  n(n-m2)  -  (n-m^n* 

that 


[P(k)  :  v  (k)] 


T(k) 

C(k) 


=  I 


(3. A. 24) 


and  T(k)  is  of  rank  n  -  m2-  Choose  ^T(k)  such  that 


TOO  = 


%(kp 


T2(k) 


5  Vk)  e  M(n-m)n 


(3.4.25) 


and 


T2(k) 
•  •  • 
C(k) 


is  of  rank  n;  thus  T^(k)  must  be  given  by 


Tx(k)  =  [Kx(k)  :  K,(k)J 


C.(k) 


C0(k) 


k  K(k)C(k) 


(3.4.26) 


where  K, (k)  e  M  ,  K„(k)  e  M  .  Partition  P(k)  into 
— l  n^m^  —2  m^m2  — 


P(k)  =  [P  (k)  :  P,(k)]  ;  P. (k)  e  M  ,  P,(k)  e  M  .  .  (3.4.27) 

1  .  — 2  —1  nm^  —2  n(n-m) 


Equations  (3.4.24)  to  (3.4.27)  imply  also  that 
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ll2(k)  ’  ^(^Kk)  +  I  <k>’ 


t20c) 


C(k) 


=  I 
— n 


(3.4.28) 


Since 


T,(k) 


C(k) 


e  M 


nn 


(3.4.28)  also  implies 

T,(k)P2(k)  =  ;  C(k)P2(k)  -  0,(M)  , 

I,«F,(k)K(kW,WV!(k)  ■  0,  ;  C(k)P,  (k)K(k)+c(k)v*(k)  =  I  . 

—2  “1  —  —2  —  — (n-mjm  —  — x  —  —  —  ~n 

(3.4.29) 

Partition  V*(k)  =  [v"(k)  1  V*(k)),  v" (k)  e  M  ,  V*(k)  e  M  ;  compatibility 
—  —1  .  I  ~1  nm^  ~c.  nm2 

implies: 

C_ (k)P1 (k)  =  0  ;  C_(k)V*(k)  =  0  ;  C.(k)V*(k)  =  I  .  (3.4.30) 

—2  —1  — m2n'l  ~  — 4  — ^  ~ m2 

Using  (3.4.30),  the  last  equation  of  (3.4.29)  can  be  reduced  to: 


C.  (k)P.  (k)K  (k)  +  C  (k) V  (k)  =0 
— 1  —1  ~2  —1  2  — m^n^ 


C1(k)P1(k)K1(k)  +  C1(k)V1(k)  =  1^ 


(3.4.31) 

(3.4.32) 


Mow  to  specify  one  explicit  minimal  order  optimum  observer  let  us 

*  w 

choose  *.P  (k)  }^_Q  such  that 


P  (k)  .  [Pj(k>  :  P2(k)]  i  Pjtk)  e  ,  P2<k)  z  Hn(n_m, 


(3.4.33) 


and  P^(k),  P^k)  satisfy: 


C(k)P;(k)  =  0  ,  ,  ;  C  (k)P  (k)  =0 

—  —2  — m' n-m)  —2  i  ^n2ml 


(3.4.34) 


Let  T„(k)  e  M.  .  ,  K,  (k)  e  M  ,  K.(k)  c  M  be  the  solution  of  the 
—2  (n-m)n  —1  m,m,  —2  ni,M. 


following: 


Cx(k)Pj(k)K2(k)  +  Cx(k)V,(k)  =  0^  m 
C.(k)P*(k)K*(k)  +  C  (k)V*(k)  =  I 

“I  — 1  —1  “1  —1  “TO, 


T2(W4(k)K,  (k)  +  X2(k>Vl(k)  =  0( 


k  =  0,  1,  2,  ...  (3.4.35) 


T*(k)P*(k)l£<W  +  I*(k)v*(k)  =  0(n.B)n_ 


T*(kjP*(k)  -  I 
—2  —1  -n-m 


Solution  for  (3.4.35)  exists  and  is  unique  .since  we  know  a  priori  that 

£  oo 

there  are  solutions  for  (3.4.29).  The  choice  of  {]>  (k)},  _n  is  nonunique, 

K.—U 

and  is  usually  chosen  so  as  to  simplify  computation.  One  minimal  order 
optimum  estimator  is  £^,*,  where 


T  (k)  = 


K  (k)C(k) 


T*(k) 


(3.4.36) 


and  K  (k) ,  T^(k)  are  given  by  the  solution  of  (3.4.35).  (Figure  3.4.) 
3.5  Detectability  and  Observability  of  Linear  Systems 

Let  us  consider  the  totally  noise  free  situation,  i.e.,  R(k)  =  0 


^.(k)  =  j),  k  =  0,  1,  ...  •  We  shall  discuss  the  notion  of  detectability  and 
observability  of  the  deterministic  system  in  terms  of  its  structural 


Fig.  3.4  STRUCTURE  OF  OPTIMUM  OBSERVER-ESTIMATOR.  THE 
NOISE  IS  ASSUMED  TO  BE  DEGENERATE  IN  GENERAL. 
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Definition  3.5.1:  The  system  3^  is  said  to  oe  detectable  at  if  for 
>c<lco)  e  Rn  arbitrary,  there  exists  an  estimator  3^  described  by 


z(j+l)  =  T(j+i)A(j)P(j)z(j)+T(j+l)A(j)V(j)x(j)+T(.i+i)B(j)uCj) 


w(j)  =  Z(j)z(j)+V(j)£(j)  ;  z(kQ)  e  Sk  =  {T(ko)ajct  e  Rn} 

o 


(3.5.1) 


such  that  w(j)  -►  x(j)  as  j  +  “.  The  system  g^  is  said  to  be  detectable 

if  it  is  detectable  at  all  k  -1,  0,  1 . 

o 

Definition  3.5.2:  The  system  is  said  to  be  completely  observable  at  k 
of  index  v  if  for  >c(k)  e  Rn  arbitrary,  we  can  deduce  .x(k)  by  observing 
£(k),  i(k  +  1),  ....  £(k  + 

One  can  easily  show  that  an  equivalent  definition  of  observability 

.  [42] 

is: 

Definition  3.5.2* :  The  system  is  said  to  be  completely  observable  at 

k  of  index  v  if  there  exists  v  <°°  such  that 
o 


Q,  -  [C’(k  ,k  )C'(k  +l).\..:*'(k  +v-2,k  )C'(k  +v-l)J  (3.5.2) 

k  ,v  ~  o  .—A  o  o  —  o  .  .—A  o  o  o 

o 


has  rank  n,  where 


i.(ijj)  k  A(i)A(i-l). . ,A(j)  ;  £  (i,i+l)  i  I  ;  i  >  j  i  k  .  (3.5.3) 


The  system  is  said  to  be  completely  observable  if  it  is  completely  ob¬ 
servable  at  all  k  with  index  v,  ,  k  =  -1,  0,  1,  ...  . 

o  o 

From  the  above  definition,  we  cannot  conclude  a  priori  any  relation 
between  detectability  and  observability  of  the  linear  system,  intuitively, 
we  may  think  complete  observability  implies  detectability  but  at  first  sight, 
this  implication  is  not  obvious.  In  this  section,  we  shall  investigate  the 
relation  between  observability  and  detectability. 


Xitbout  less  of  esaraliev,  ws  cas  sec  k  =  0,  ana  for  simplicity  write 
"  o 

5V  *  0  =  {C' {0V.il f©,Q)C* Cl);.. -i.: (R-2,0)C' eh-!) ]  .  (3.5-4} 

se  s£sll  assure  chat  A(R)  is  invertible  for  all  k  =  — ,  -1,  0,1,  —  -  Le 
cs  ceMte  for  j  -  i: 


^(i,j)  *  =  A_‘{j)  ’i  Cj);iA(i,i  -  I)  4  ^ 


(3.5.5) 


If  me  assure  sere  a  priori  distribution  ea  the  initial  condition  of  jc(0>, 
then  tre  can  make  use  of  the  results  in  Section  3.3  to  obtain  the  equation 
for  the  error  covariance,  this  is  given  by  [see  (3-3.11),  (3.3.12)] 


1'tkhl)  =  _~(k)  -  V  <k*!)C(fchl):~(fc)  ;  k  -  0,  1, 


2*(0)  =  :  -  :  c* (0) (ccc)~  c1 (0))~1c(0>: 


(3.5.6) 


where 


1  (k)  =  A(kK  (k)A'  (k)  :  k  =  1,  2, 


(3.5.7) 


V  (k)  r  U.  (k-D)  *  -V  £  M  V[C(k)r(k-l)C’(k)]  =  L  (k-l)C’ (k)> 

rc“i  tin  **” 

It  -  o,  i,  ... 

‘S+  *90 

Tr.eoren  3*5.3:  Let  :Z  (k)}  be  a  sequence  satisfying  (3.5.6)  and  (3.5.7) 
-  —  o 

with  V* (k)  t  T^Ck  -  1)).  If  0  (but  arbitrary),  then  the  null 

J- 

soace  of  _Z  (k)  equals  to  the  range  space  of  ^(O.k  -  k  =  0,  1,  ....  , 

Proof:  We  shall  use  induction  on  k.  For  k  =  0,  we  have  from  (3.5.6) 


:  (0)^  =  i  fo)c*  (0)  =  (0)  -  z^c*  (0)  =  o 


(3.5.8) 
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Thus  we  conclude  that 

N(£X(C))  3  R  (Q  )  ;  dim  N(I*(0))  >  dim  R  (Q  )  =  m  (3.5.9) 

—  a  — o  —  a  o 

where  R  (*)  denotes  the  range  space.  By  assumption,  Z  >0,  thus  from 

3.  O 

(3.5.6)  we  have 

N(z*(o))n  n(z  c'(o)(c(o)?:  c,co))''1c(o)e  )  =  {0}  (3.5.10) 

—  — 0~  —  —  *“  — o  — 

C(0)  is  of  range  it,  therefore  (3.5.10)  implies 

m  >  dim  N(Z*(0))  .  (3.5.11) 

Equation  (3.5.9)  and  (3.5.11)  imply  that 

N(:*(0))  =  Ra(Qo)  =  RQ^O.-l)^)  .  (3.5.12) 

Let  us  assume  that  for  k  =  i,  we  have 

N(Z*(i))  -  Ra0^{Q,i  -  1)^)  .  (3.5.13) 

From  (3.5.4)  and  (3.5.5) 

(A_1(i))'l^(0,i  -  1)^  =  ^(O.i)fi.  .  (3.5.14) 

Let  v^  c  Ra(Ji^(0,i)Q^) ,  then  there  exists  some  x  £  Rn  such  that 

v  =  ^(O.i^x  =  (A“1(i)),J^(0,i  -  l)&.x  (3.5.15) 

and  so  A/ (i)v  e  Ra(j[>^(0,i  -  1)C^),  and,  by  the  induction  hypothesis,  we  also 

have  A' (i)v  e  N(E  (i))  or  v  e  N(A  (i)).  By  (3.5.6),  we  conclude  v  £  N(E  (i  +  1)). 

* 

Also,  by  compatibility,  the  null  space  of  Z  (i  +  1)  includes  the  range  space 
of  C' (i  +  1).  Combining  the  two,  we  have 
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N(r*(i  +  1))  •  (3.5.16) 

We  also  have  rhu  inequality 

din  N(Z*(i  +  1))  1  dir  R  (*'  (0,i)£.  )  .  (3.5.17) 

3  i\  It  1 

Let  S.  =  {v  s  Rn!v  s  R  (£  (i))  OS(C(i  +  l)A(i))}.  Since  Rn  is  finite 
x  —  -  a  “  —  — 

dimensional,  from  the  induction  hypothesis  (3.5.13),  we  have 

R  (I  (i))  =  N(Q!o..  (0,i  -  1)).  Therefore  any  v  e  S.  is  described  by 
3  A  X 

C(i+l)A(i)v  =  0  :  fi^(0,i-l)v  =  0  =— ^+J^(0,i)A(i)v  =  0  .  (3.5.18) 

Since  by  assumption  A(i)  is  nonsingular,  equation  (3. 5. IS)  implies  that 

dim  S±  =  dim  N(^+1iA(0,i))  =  n  -  dim  Ra(^(0,i)£i+1)  .  (3.5.19) 

Let  be  the  image  of  through  the  transformation  £  (i)A' (i) ;  i.e., 

S  ^  =  {w  c  Rnj_I  (i)A'  (i)w  =  v  ♦  v  £  S^}.  Let  be  the  subspace  which  is 

— 1  *  -1 

equal  to  modulo  the  null  space  of  Z_  (i)A  (i).  has  the  same  dimension 

as  S . ,  and  so 
x 

dim  S.^  =  n  -  dim  R  (>j>I(0,i)Q  .)  .  (3.5.20) 

i  a  “A  l+l 

Now  let  w  £  S.\  w  0;  then  from  the  definition  of  S.  and  S.\  we  have 

l  —  li 

C(i  +  l)£*(i)w  =  0  ;  A*(i)w  /  0  .  (3.5.21) 

From  (3.5.6),  we.  conclude 

Z  (i  +  l)w  -  &_  ( i)w  /  £ 


(3.5.22) 
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*  -1 

Therefore,  the  null  space  of  I  (i  +  1)  and  the  space  have  only  the  zero 
element  in  common;  thus 

dim  N(E  (i  +  1))  In-  dim  S  *  *=  dim  Ra(i! (0,i)^.+^)  .  (3.5.23) 

Equations  (3.5.16),  (3.5.17),  and  (3.5.23)  imply 

N(Z*(i  +  1))  =  ^(^(0,1)2^)  .  (3.5.24) 

The  theorem  follows  from  induction. 

A  direct  consequence  of  the  above  theorem  is  the  following  result  : 
Theorem  3.5.4:  Let  be  completely  observable  at  time  k  of  index  then 
there  exists  an  optimum  observer  &,j,,  T  e  ?v»  which  will  reconstruct  the 
exact  state,  x(j),  in  at  most  steps  (i.a.,  at  time  k  S  j  -  k  +  v^). 

Thus  if  is  completely  observable  at  time  k  of  index  v^t  then  is  de¬ 
tectable  at  time  k;  if  is  completely  observable,  then  is  detectable. 

Theorem  3.5.4  generalizes  Kalman's  results  [41]  in  deadbeat  estimators 
for  this  reason,  we  may  refer  to  st  .n  an  optimum  observer  ,  T  £  <LA,  as  a 
deadbeat  observer.  Clearly  there  is  more  than  one  class  of  deadbeat  ob- 
parameterized  by  {VG;)}”_q,  V(k)  e  (k  -  1),  k  =  1,  2,  ...  . 

Among  these,  we  shall  find  the  simplest  deadbeat  observer. 

Theorem  3.5.5:  Let  S.  be  completely  observable*,  the  class  of  minimal  order 

J 

ieadbeat  observers  is  of  order  n-m  . 

Proof :  Clearly,  the  class  of  minimal  order  deadbeat  observer  must  be  of 
order  greater  than  or  equal  to  n  -  m.  To  prove  the  theorem,  we  need  to 

CO  A  & 

find  a  sequence  {V^(k)}^_Q,  V^(k)  c  \-i/—  (k  -  1)),  k  =  1,  •••  such  that 

the  matrix  (I  -  V,(k)C(k))  has  rank  less  than  or  equal  to  n  -  m  for  all 
n  — 1  — 

k  =  1  9 

Is  JL  )  4.  )  • 
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Construct 

V*(k)  =  A*(k-I)C'  (k)  (C(k)A* (k-l)C'  (k))^  -f-  C*  (k)  (C(k)C'  (k))"1  • 

(3.5.25) 

(I  -  C(k)A*(k-l)C'(k)(C(k)A*(k-l)C,(kli?}  k  =  1,  2,  ... 

— ra  —  —  —  —  —  — 

a 

where  M  denotes  the  pseudoinverse  of  a  matrix  M.  Using  the  properties  of 
pseudoinverse  (see  Appendix  A)  we  have 

V*(k)C(k)A*(k-l)C'  (k)  =  A^k-DC'  (k)  (C(k)A*(k-l)C’  (k))^C(k)A*(k-l)C'  (k) 

=  VX(k) (C(k)A*(k-l)C* (k)) (C(k)A*(k-l)C' (k)/c(k)A*(k-l)C' (k) 

=  V*(k)C(k)A*(k-l)C'(k)  =  A*(k-i)C* (k);  k  =  1,  2,  ... 

(3.5.26) 

.4  * 

Therefore  V^k)  e  ^-1^—  (k  -  1)),  k  =  1,  2,  ...  .  From  (3.5.25),  we  deduce 

C(k) (I  -  V*(k)C(kJ  =  C(k)  -  C(k)  =  0  .  (3.5.27) 

—  — n  —x  —  —  —  — 

Since  _C(k)  is  of  rank  m,  (3.5.27)  implies  that 

rank  (I  -  V*(k)C(k))  5  n  -  m  (3.5.28) 

~ n  ~1  - 

and  the  theorem  follows. 

Finally,  we  would  like  to  derive  a  test  for  the  detectability  of  linear 
systems.  Using  (3.3.7),  we  have  easily  the  following: 

Theorem  3.5.6:  A  system  5^  is  detectable  if  and  only  if  there  exists  a 

00 

uniformly  bounded  sequence  (V(k)}^_Q  such  that 

-aJi-j  | 

11*3  (i.j)ll  ±  axe  .  >  0  (3.5.29) 
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where  for  i  -  j : 


6,  (V(k+1))  £  A(k)  -  V(k+l)C(k+l)A(k)  ;  6,  (i,j>  k  6. (V(i+l))e.  ,(VCi)> 

— Jr  —  —  -  -  —  jL  — 


...  e.(v(j+i)) 

-j 


(3.5.30) 


3.6  Asymptotic  Behavior  of  Optimum  Estimators 

In  this  section,  we  shall  investigate  the  asymptotic  behavior  of  an 
2 

optimum  estimator  T  z  3-*,  for  the  stochastic  system  S^.  shall  say 
that  the  system  is  detectable  if  its  deterministic  correspondence,  8^, 

is  detectable o  The  investigation  is  carried  out  by  considering  the  minimal 

*  00  _  . 

Riccati  sequence  (Z^  (k) } ^_q  which  describes  the  evolution  or  the  minimum 

error  covariance. 

First,  let  us  assume  that  the  initial  time  is  kQ,  and  consider  the 

•k  k 

behavior  of  Z  (k),  as  k  -*•  °°,  where  Z_  (k)  satisfies 


I*(k+1)  =  A*(k)  -  V*(k+l)C(k+l)A*(k) 


(3.6.1) 


E*(k  )  -  I  -  Z  C'(k  ) (C(k  )E  C'(k  )  +  Q(k  )]~lC(k  )Z 
—  o  — o  — 0“"  O  —  O  ”0 —  o  o  —  O  “O 

and  V  (k)  z  ^  (Z_  (k  -  1) ) ,  k  =  kQ  +  1 ,  k  +  2 ,  ...  . 

k 

Vteorem  3.6.1;  The  minimum  covariance  error  Z_  (k) ,  will  remain  bounded 
for  all  k  =  kQ  +  1,  k  +  2,  ...  if  and  only  if  the  system  8^  is  detectable. 
Proof:  If  is  detectable,  then  from  theorem  3.5.6,  we  note  that  there 

CO 

must  exist  a  uniformly  bounded  sequence  { V_Ck) } ^_q  such  that  the  resulting 
solution  of  (3.3.7)  (with  kQ  replacing  0)  will  remain  bounded  for  all 

*  03 

k=k,k  +  1,  ...  .  Since  {L  (k)}.  n  satisfying  (3.6.1)  is  the  minimal 

O  O  K“U 

sequence  with  respect  to  the  solution  set  of  (3.3.7)  we  conclude  that 

> 

k 

Z  (k)  must  also  be  bounded  for  all  k  =  k  ,  kQ  +  1,  ...  . 
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Conversely,  if  Z_  (k)  remains  bounded  for  all  k  =  kQ,  k  +1,  . then 
from  (3*3.7),  we  have 

li£s  a^a.,  >  0  (3.6.2) 

v* 

and  so  S7  is  detectable. 

Next,  we  shall  assume  that  the  present  time  is  k,  and  assume  that  the 

Initial  time  k  -*• 
o 

Let  us  rewrite  (3.3.9)  in  a  more  suggestive  form:  (set  k^  =  kQ  -  1) 


I  (k+l,k’ ;I  )  =  (A(k)-V(k+l)C(k+l)A(k))Z  (k+i,k‘;Z  ) (A(k)-V(k+l)C(k+l)) ! 

— V  o  — o  —  —  —  —  — V  o  — o  —  —  — 

+(I  -V(k+l)C(k+l))R(k)(I  -V(k+l)C(k+l)),+V(k+l)Q(k+l)v’(k-fl);  k  «  k'.k'+l,. 
— n  —  —  —  — n  —  —  —  ~  o  o 


:  (k’,k';Z  )  =  :  .  (.3.6.3) 

— v  o  o—o  —o 


As  we  have  noted,  (3.6.3)  is  the  same  as  (2.5.1)  except  for  seme  changes  in 
the  symbol  (3.3.14),  (3.3.15).  We  shall  still  use  the  symbol  as  defined 
by  (2.5.2)  with  the  obvious  change  (3.3.14).  As  usual,  we  shall  denote  the 
minimal  Riccati  sequence  with  respect  to  the  solution  set  of  (3.6.3)  by 
J-—  (k,ko;-o)  ;k=k' ' 

^  *  » 

Lemma  3.6.2:  There  exists  an  unique  bounded  sequence  \Z_  (k;_0_) }|^__eo  such 

that 


lim  Z  (k,k';()) 

k’—»  ° 

o 


=  Z  (k;£)  for  all  k 


(3.6.4) 


and  Z  (k;0)  satisfies 


-  'k 


Z  (k+l;0)  =  vk(V  (k+1) ,Z  (k;0))  ;  V  (k+1)  t  hk(Z  (k;0))  (3.6.5) 


if  and  only  if  the  system  §2  is  detectable. 
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Proof:  Let  us  denote 

9,  (V)  4  A(k)  -  V  C(k+l)A(k)  ;  k  =  k* ,  k'  +  1,  . . .  .  (3.6.6) 

k  —  —  —  —  o  o 

Using  lemma  2.5.2  and  Equation  (3.6.3),  we  have  the  inequality 

9.  (V* (k,k’ -1 ; 0))(E* (k-1  , k  * -1 ; C) -I* (k-1 ,  k ' ;  0) ) 6/  (V*(k,k’-1;0))“ 

— k-1  —  o  —  —  o  —  —  o  —  — k-1  —  o 

5  Z*(k,k’-1,0)-E*(k,k* ;0)  .  (3.6.7) 

O  O 

Since  Z  (k^,k^-l;(J)  >  ()  and  Z  (k^,k^;f))  =  0 ,  (3.6.7)  implies  that 

Z  (k,kQ-l;())  -  Z  (k,kQ;jO)  for  all  k  1  kQ  .  (3.6.8) 

•k 

If  *s  detectable,  then  by  using  theorem  3.6.1,  Z  (k,kQ;0)  will  be 

bounded  for  a  fixed  k  and  all  kQ  1  k.  By  the  monotone  convergence  theorem 

of  nonnegative  operators  [32],  we  conclude  that  there  exists  an  unique 
* 

E  (k;0)  such  that 

lim  E*(k,k’;0)  =  Z*(k;0)  >  E*(k,k  ;0)  ;  k  >  —  .  (3.6.9) 

.  ,  —  O’—  -  —  O  —  o 

o 

Let  us  define 

f*(k+l;0)  =  ^k(V*(k+l),E*(k;0>  ;  V*(k+1)  e  Uk(I*(k;0))  .  (3.6.10) 

By  lemma  2.5.2  and  (3.6.8)  we  have  for  all  k^  > 

f  (k+1 ;  0) -Z  "  (k+1 ;  0 )  S  ( V*  (k+1 )  ,  I*  (k ; 0) ) (V*  (k+1 , k^ ;  0 )  J?  ( k ,  k ^ ;  0) ) 

-  4 (1  (k+1 > k^ ; 0)  ,  E  * (k; 0)  ) (V*  (k+1 ,  k^ ; 0) , Z*  (k , k^ ; 0) )  .  (3.6.11) 

7V*(k,k’-i;0)  e  li  . (Z*(k-l,k’-i;0))  ,  i  =  0,  1 

—  o  K-l  —  o  ~ 


(3.6.18) 


c 


t  I 

t  1 
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Since  Z  (k’jk'jE  )  =  Z  -  0,  then  (3.6.18)  implies  that 
~  o  c  - o  — o  - 

0  <  E*(k+l,k’  ;Z  )  -  Z*(k+l,k’;Q)  1  (k(k,k')I  iKk.k') 

—  —  O  — O  —  O  "“tJ  O  O  v  o 


where 


.t0(k,k^)  £  9k(V#:(k,k;;0))ek_1(V  (k-l,k-0)...4f(V  (k^.k^O)  .  (3.6.20) 


If  the  system  is  detectable,  then  by  theorem  3.6.1,  we  must  have 


I  li8(k»ko)  i  I  - 


-“2 1 k-Ko 1 


al,a2  >  ° 


(3.6.21) 


and  so  using  lemma  3.6.2  and  equations  (3.6.19),  (3.6.21),  we  have 

lim  Z  (k , k ' ; I  )  =  lim  Z  (k,k';0)  =  Z  (k;0)  k  Z  (k)  .  (3.6.22) 

k'-^-oo  °  k '->-«>  ° 

O  O 

Equation  (3.6.17)  follows  from  (3.6.5). 

The  proof  in  the  reverse  direction  is  the  same  as  in  proving  lemma 
3,6.2. 

Finally,  we  shall  consider  the  time  invariant  case  where  A,  £,  C£,  R 
are  constant  ar.d  bounded  matrices.  In  this  case 


Z*(k,k’;Z  )  =  ZX(k-k',0;E  ) 
—  0  -0  —  o  •-  — o 


(3.6.23) 


thus  taking  k^  =  0  is  th-,  same  as  considering  k  ->  °°.  We  shall  only  con¬ 
sider  k'  =  0,  and  k  -*■  ,a. 
o 

sV 

Theorem  3.6.4:  There  exists  a  bounded  Z  such  that 


f 

? 

L 


lim  E*(k,0;E  )  =  Z* 

k-H« 


Lnd  Z 


(3.6.24) 


satisfies  the  algebraic  equation 
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JU  j. 


l“  =  A  -  V'C  A"  ;  jf  V(E ‘)  I  (V  £  M  |V(C  A  A’  +  fi)  =  AC'} 

(3.6.25) 

4*  =  A  Z*A'  +  (3.6.26) 


if  and  only  if  the  system  Sp  *s  detectable. 
Proof:  In  the  time  invariant  case 


&GH)  -  4+1(v,E)  i  jSl(v,e)  ;  e^(v)  =  ^(v)  g  e(V)  .  (3.6.27) 


■Tt+l 


Using  lemma  2.5.2,  we  have 


E*(k+1,0;0)  -  l*(k,0;0)  >  6(V*(k,0;0)) (Z*(k,0;0)  -  (k-1 , 0 ; 0) ) 


0’(V  (k,0;0)) 


(3.6.28) 


Since  Z_  (1,0;0)  t  Q,  (3.6.28)  implies  that 


jf(k+l,0;0)  >  l  (k,0;0)  k  =  0,  1,  2,  ... 


(3.6.29) 


By  theorem  3.6.1,  _£  (k , 0 ; C))  will  remain  bounded  if  and  only  if  S,;  is  de- 

"k 

tectable,  and  so  by  (3.6.29),  one  concludes  that  there  exists  l_  sach  that 

lim  E*(k,0;0)  =  Z  .  (3.6.30) 

k-«o 

j|c 

Using  (3.6.23)  and  lemma  3.6.2,  S  satisfies  the  algebraic  equation 


A  A 

I  =  ,1  )  =  A  -  vc  A 


•ie  -> 

V  e  \s(Z  ) 


(3.6.31) 


A 

and  i  is  given  by  (3.6.26)  if  and  only  if S p  is  detectable.  Using  theorem 
3.6.3  we  have  the  desired  results. 

Theorem  3.6.5:  If  Sp  is  detectable,  there  exists  only  one  nonengative 
£ 

definite  matrix  Z  which  satisfies  (3.6.25),  (3.6.26). 
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Proof:  Let  us  define 

Z°  =  0  ;  V°  e  V(0) 

and 

* 

Z1  =  tKV1-1,!1"1)  ;  V1  e  V(£x) 

By  (3.6.30),  such  a  constructed  sequence  of  Z1  will  converge  to  1 
satisfies  (3.6.25),  (3.6.26). 

Let  1-0,  and  %_  satisfies  (3.6.25),  (3.6.26);  i.e., 

I  =  1<V,1)  ;  V  e  V(Z) 

3y  lemma  2.5.2,  we  have 

e_(v)  (|  -  z1”1)!'  (v)  <  I  -  z1  <  _o  (v1)  (|  -  i1-1)e/  (v1) 

By  construction,  Z°  =  (),  thus 

0  -  I  ~  I1  -  ^(i,0)|^(i,0) 

where 

(i,0)  £  I(Vl)e(V1-1)  ...  6(2°) 

If  §2  is  detectable,  then 

-aji-j  1 

I  |±q  (i ,  j )  |  |  ±  o^e  >  0 

thus  we  have 

vi  = 

Z  ->■  Z  as  l  +  “ 


(3.6.32) 

(3.6.33) 
,  which 

(3.6.24) 

(3.6.35) 

(3.6.36) 

(3.6.37) 

(3.6.38) 

(3.6.39) 


and  uniqueness  follows. 


1 


3.7  General  Discussion 

la  this  chapter,  ve  have  cbtaiaed  an  optima  unbiased  estimator  for 
the  stochastic  system  5.,  vhere  the  observation  \oise  nay  be  degenerate 
(0(k)  1  0)  or  singular  {Clk;  ~  0).  In  essence,  the  optima  estimator  is 
specified  by  the  relations: 


z* (fci-1)  -  7(fc5-l)A(k)F(k)z* (k)-7(fcri>A(k)V* (fc)v(k)  *T<k*l)B(k)u(k) 


(3.7.1) 


-4k)  =  Fik)z  (fc)  -V  (k)y(k)  ;  z(0)  =  T(0)x 


sh.'re  s  (k)  is  the  optical  estimates  of  x{k) .  (See  Figure  3.4.)  The  matrices 
j?(k),  T(k)  satisfy 


?(k)T(k)  +  V  (k)C(k)  =  I 


(3.7.2) 


and  V  (k)  is  given  by  (see  theorem  3.3.1) 


V  (k)  s  U__xu  (k-D)  ,  fc  =  i,  2,  ...  ;  v  (0)  =  :  C' CO)  tCWG^C1  <0)~Q(0) } 


7*(k+l)  =  ~*(k)-V*(k-rl)C(k+lK~(k)  ; 


(3.7.3) 


:(0)  =  :  c’  (0)  ic(Q):  c*  (O)-hhq)  1  c(o>: 

—  — o  c  —  -  o  —  — t 


k  (k)  =  A(k):  (k)A(k)eR(k) 


Note  that  t V  (k);n_n  can  be  precomputed  when  the  structure  of  5-  and  the 

..  k  x 

statistical  law  of  the  uncertainties  are  known.  In  general,  -V  (k) -j._q 

may  not  be  unique.  In  the  special  case  when  (£(k)  >  0  or  C(k+l)R(k)C' (k+1)  >  0 

„  OC 

we  have  uniqueness  in  -V  (see  theorem  3.3.1). 

,  ^  3k  *X  t 

Once  <V  (k)  is  found,  we  can  choose  different  ! P^(k)  and 

% 

•T(k);^_Q  such  that  (3.7.2)  is  satisfied,  and  so  one  can  construct  different 


optlnus  estimators  £.“  where  the  dimension  of  the  observer  state  vector 

T 

z(k)  are  different  depending  on  the  choice  of  {T(k)},_g,  T(k)  t  **y*fl-)* 

«•% 

It  has  been  shown  that  in  the  special  case  when  R(k)  •*  0,  the  minimal 
order  optinua  observer  is  of  dicension  n  -  where  a is  the  number  of 
noise-free  channels  available.  Though  the  proof  is  given  for  the  special 
case,  it  is  conjectured  chat  the  results  will  be  true  in  the  general  case 
when  R(k)  -  0  or  even  R(k)  =  <0. 

* 

Finally,  the  asymptotic  behavior  ol  Z  (k)  given  by  (3.7.3)  was  con¬ 
sidered  in  great  detail.  Necessary  and  sufficient  condition  were  derived 
* 

for  (k)  to  be  uniformly  bounded  and  existence  of  its  steady  behavior. 

In  the  following,  we  shall  discuss  some  of  the  relevant  points  in  the 
development  of  this  chapter. 

(A)  Discussion  of  Approaches 

Different  approaches  are  available  to  filtering  problems.  The 
Projection  approach  was  used  by  Kalman  to  first  obtain  the  Kalman  filter. 

The  starting  point  of  this  approach  is  the  Projection  Theorem  (Theorem 
3.3.2).  There  is  also  the  Baysian  approach  [43]  where  one  computes  the 
conditional  expectation  of  the  state,  x^(k) .  Also,  a  max-likelihood 
approach  [44]  is  available  to  filtering  problems.  Then,  there  is  the 
approach  of  unbiased  minimum  error  covariance  estimates  [10],  and  of 
weighted  least  square  estimates  [43].  In  the  linear-Gaussian  case  all 
thest  approaches  will  yield  the  same  solution  (see  also  section  3.3).  It 
is  hard  to  argue  which  of  the  above  approachs  to  the  problem  is  more  funda¬ 
mental  than  the  other,  for  this  highly  depends  on  one’s  philosophical 
viev;point  to  the  problem.  One  may  argue  that  the  Baysian  approach  is  the 
most  fundamental  approach.  This  is  true  to  the  extent  where  one  can  justify 
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the  knowledge  on  a  priori  distribution  of  all  the  underlying  random 
vectors. 

The  approach  used  in  this  chapter  seems  to  be  a  new  approach  to 
the  problem  where  one  starts  from  deterministic  consideration  This  is 
true  in  some  sense.  If  the  knowledge  on  the  a  priori  distribution  of  the 
state  ^c(O)  is  correct,  then  the  approach  is  equivalent  to  that  of  unbiased 
minimum  error  covariance.  To  verify  this  statement  let  us  consider  the 
stochastic  system  (with  ju(k)  i  0) .  We  look  for  an  unbiased  estimator 
which  is  nonanticipative.  In  general,  such  •'n  estimator  is  described  by  [45} 

z(k+l)  =  F(k)z(k)  +  G(k)^(k)  ;  z(k)  t  RS 

£:  •  (3.7.4) 

w(k)  *  P(k)jj(k)  +  V(k)v(k) 

The  initial  condition  of  jz(0)  is  some  linear  transformation  of  x^;  i.e., 

z(0)  =  T(0)x  (3.7.5) 

—  —  — o 

and  for  all  k  1  0,  we  want  E{w(k)}  =  E{x(k)}.  With  this  restriction  we 
have 

(P(O)T(O)  +  ^(0)0(0)^  =  .  (3.7.6) 

We  would  like  to  construct  the  estimator  completely  independent  of  the 
mean  of  x.(0)  >  then  (3.7.6)  implies 

P(0)T(0)  +V(0)C(0)  =  1^  (3.7.7) 

and  so  we  must  have  s  '  n  -  m  and  T(0)  z  (jC (0)  ;m,s ,n) .  For  k  >  0,  the 
unbiased  restriction  gives 
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iKkJl^Ck-l.OJTCO)^  +  22  oF(k-l,i+l)G(i'C(i)jA(i-l,0)xo] 


+  V(k)C.(k)o (k-I,0)x  =  c. (k-l,0)x 
—  —  ~A  - o  ~A  ~ o 


(3.7.8) 


If  A(k)  are  invertible  for  all  k,  and  the  structure  of  the  estinator  is 
independent  of  x  ,  then  (3.7.8)  implies  that 

"  O 

k-1 

P(k)[;F(k-l,0)T(0)vA(0,k-l)  +  22  o.F<k-l,i+l)G(i)C(iKA(i  ,k-i)  ] 


+  V(k)C(k)  =  I 

—  —  Tl 


(3.7.9) 


Define 


T(k)  =  Cp(k-l,0)T(0)vA(0,k-l)  +22  £F(k-l,i+l)G(i)C(i)vA(i,k-l)  .  (3.7.: 

i=0 

Then  T(k)  •;  .'.(C(k)  ;n,s,n)  and  T(k)  satisfies: 

T(k  +  1)  =  F(k)T(k)A_1(k)  +  D(k)C(k)A_1(k)  .  (3.7.11) 


Such  an  estimator  can  be  realized  by  picking 


F(k)  =  T(k  +l)A(k)P(k)  ;  D(k)  =  T(k  +  l)A(k)V(k)  .  (3.7.12) 


Comparing  with  theorem  3.2.3,  we  see  that  all  unbiased,  nonanticipative 

2 

estimators  can  be  realized  by  an  observer  T  e  3T  and  its  associated 
2 

estimator  Therefore  , the  restriction  of  using  an  observer  and  its 

associated  estimator  as  an  estimating  device  is  the  same  as  restricting 
ones  attention  to  bnly  unbiased  state  estimators. 

But  if  the  a  priori  assumption  on  x  is  different  from  the  true  mean 

-  Q 

of  >:(0) ,  then  it  is  not  unbiased  minimum  mean  square  error  approach.  In 
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fact  in  this  situation,  nearly  all  the  approaches  mentioned  above  may  not 
be  justified.  But  we  shall  see  in  discussions  that  under  some  mild 
conditions  our  approach  is  still  valid  even  with  incorrect 
on  a  priori  distribution  on  the  initial  state  x(0) . 

(B)  Dimension  of  Observers 

From  the  point  of  memory  storage,  we  would  like  to  find  the  minimal 
order  optimum  observer;  but  from  the  point  of  view  of  computation,  one 
may  not  want  to  find  the  minimal  order  optimum  ooserver.  One  may  want  to 
look  for  those  observers  G^,  T  e  where  the  number  of  nonzero  entries  of 
T(k  +  1)  A(k)  £(k),  T(k  +  1)  A(k)  V(k) ,  and  £(k)  is  kept  to  a  minimum.  No 
systematic  way  of  finding  such  observers  is  available;  in  general  this  will 
depend  on  the  specific  problem  under  consideration. 

(C)  Detectability  and  Observability 

Detectability  is  a  weaker  condition  than  observability  (see  theorem 
3.5.4).  Essentially,  detectability  implies  that  in  noise  free  situations, 
one  can  deduce  the  current  state  (but  not  the  initial  state)  of  the  system 
if  given  infinitely  long  observation  period,  and  so  it  is  not  the  same  as 
"asymptotic  observability"  (if  such  a  concept  can  be  defined).  In  all 
sequential  estimation  problems ,  one  is  interested  to  estimate  the  current 
state  rather  than  the  initial  state  of  the  system,  so  one  would  expect 
that  detectability  would  be  the  intrinsic  property  which  will  assume  nice 
behavior  of  the  minimum  error  covariance  when  noises  are  present.  This 
physical  intuition  was  verified  in  section  3.6.  We  showed  that  detecta¬ 
bility  of  linear  system  gives  the  necessary  and  sufficient  condition  for 


This  viewpoint  is  due  to  F.C.  Schweppe 
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uniformiy  bounded  ±  (k)  and  the  existence  of  its  steady  state  behavior. 
Observability  implies  that  in  the  noi,se  free  situation,  we  can  deduce 
the  initial  state  of  the  system  if  given  a  long  enough  observation  period. 

Of  course,  knowing  the  initial  state  will  enable  us  to  deduce  the  current 
state;  but  as  long  as  sequential  estimation  is  the  goal,  the  knowledge 
of  initial  state  will  be  nice  but  not  absolutely  necessary.  Except  in  the 
smoothing  estimation,  where  we  are  interested  in  finding  not  only  the 
current  estimate,  but  the  whole  trajectory  estimate;  thus  in  this  situa¬ 
tion,  detectability  may  not  be  enough  to  assure  the  "nice  behavior"  of 
* 

Z_  (k) ;  we  need  observability  of  the  system. 

In  the  development,  we  assume  an  a  priori  distribution  on  the  initial 

state  x(0) .  This  assumption  can  only  be  justified  if^as  time  advances 

and  information  accumulates,  the  resulting  performance  will  be  independent  of 

the  a  priori  distribution  of  x(0).  Assume  that  the  true  mean  of  x(0)  is 

x'  but  we  guess  its  mean  to  be  x  .  Since  the  mean  of  the  state  of 
—o  -o  2 

satisfies  (a.s.)  the  deterministic  equation  described  by  (see  section 

2.3),  then  detectability  implies  that  even  with  a  wrong  assumption  on  the 

mean  of  ;;(0) ,  the  optimum  observer  will  give  an  asymptotically  unbiased 

■ k 

estimate  Thus  as  k  +  »,  I  (k)  truly  represents  the  error  covariance. 

From  theorem  3.6.3,  we  see  that  in  the  steady  state  period,  the  error 
covariance  is  independent  of  the  covariance  of  _x(0).  Therefore  if  is 
detectable^  the  performance  will  "merge"  when  information  accumulates  even  if 
we  started  off  with  different  assumptions  on  the  statistical  law  x(j). 

Thus  detectability  justifies  the  assumption  on  knowing  the  mean  and  co- 
variance  of  x(0) . 
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(D)  Secuentiallv- Correlated  Observation  Noise 

The  derived  results  are  also  applicable  to  the  case  when  the  observa¬ 
tion  noise  satisfies:  [see  equation  (3t3-i)  forS] 


n(k  -  i)  =  A(k)n(k)  -5-  ,(k)  ;  n(k)  t  R1- 


(3.7.13) 


SC  X 

where  {v.(k)},._ ^.(0),  x(G),  and  are  independent  with  statistical 

K"U  r  1-1  tv'  v 

law  (3.3.2),  (3.3.3)  and 


2(0)  G(V^> 


Y(k)  G(0,90O> 


(3.7.14) 


We  can  define 


X3(k)  5 


;  Aa(k) 


£a(k)  =  ...  ;  2a(k)  =  ... 


&  J 


(3.7.15) 


Then  we  have  the  augmented  system 


xa(k  +  1)  =  AS(k)  +  S3(k)u(k)  +  £a(k) 


X(k)  =  [C(k)!l  ]x  (k) 
—  .~m 


(3.7.16) 


We  can  apply  the  derived  results  to  the  above  system  Note  that 
xa(k  +  1)  z  Rn+m,  but  since  the  noise  free  observation  is  of  dimension  m, 
the  minimum  order  optimum  observer  is  of  order  n.  This  problem  has  also 
been  considered  by  Henrikson  [46],  Bryson  and  Ho  [43]  using  a  different 


approach.  We  can  easily  verify  that  the  results  obtained  by  applying  the 


*» 
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derived  results  to  this  special  class  of  problen  ;  thev  are  the  sanie  as  those 
obtained  by  Henrikson.  This  special  application  will  be  considered  in  a 
future  investigation. 

3.3  Perspective 

Observers  for  a  linear  system  were  introduced  by  Inenberger  [35], 

P6J.  He  only  considered  continuous .linear , tine  invariant  systems.  Ob¬ 
servers  for  discrete, linear, tine  invariant  systems  were  discussed  by  Aoki 
and  Huddle  [37]  in  relation  to  a  constrained  estimator  problen.  Observers 
for  discrete  linear  tine  varying  system  were  first  introduced  and  studied 
by  Tse  and  Athans  [38]. 

Optimum  linear  filtering  for  discrete  linear  time  varying  systems  was 
investigated  by  Kalman  139],  [40]  using  the  projection  theorem  approach. 
Deadbeat  estimator  for  discrete  time  invariant  system  were  derived  by  Kalman 
[41].  The  unbiased  approach  to  optimum  linear  filtering  problems  was  used 
by  Athans  and  Tse  [10],  Tse  and  Athans  [38];  the  unbiased  approach  to  non¬ 
linear  filter  was  used  by  Athans,  Wishner,  Bertolini  [42]. 

Detectability  was  first  introduced  by  Bonham  [32]  as  the  dual  concept 
of  stabilizability.  Detectability  as  defined  by  definition  3.5.1  seems 
to  be  more  appropriate  and  more  general  than  that  of  Wonham's  (U’onham  con¬ 
sidered  only  the  time  invariant  case). 

The  asymptotic  behavior  of  minimum  error  covariance  for  discrete 
linear  systems  were  not  investigated  in  full  detail  in  the  current  litera¬ 
ture.  Deyst  and  Price  [28],  Sorenson  [29]t  and  Aoki  [30]  considered  to 
some  extent  the  asymptotic  properties  of  the  minimum  error  covariance.  They 
confine  themselves  to  consider  the  special  case  when  the  observation  noise 
is  regular  (£(k)  ^) .  Little  or  no  attention  is  paid  to  the  case  when  the 


The 


observation  noise  is 
treatment  in  section 


degenerate  (0(k) 
3.6  is  original. 


2  0)  or  singular  (£(!:)  =  0)  . 
antj  consider-  all  different  cases 


in  a  unifying  manner 


CHAPTER  IV 


OBSERVER  THEORY  FOR  CONTINUOUS  TIME  LINEAR  SYSTEMS 

4.1  Introduction 

The  problem  of  state  estimation  for  discrete  linear  systems  was  con¬ 
sidered  in  detail  in  Chapter  III.  J.n  this  chapter,  we  shall  consider  the 
state  estimation  problem  for  continuous  time  linear  dynamical  systems. 

Aside  from  the  fact  that  state  estimation  is  of  prime  importance  in  the 
design  of  optimal  control  systems, the  problem  in  itself  is  of  great  impor¬ 
tance  in  the  design  of  modern  communication  systems. 

The  structure  of  this  chapter  is  as  follows.  In  section  4.2,  we  con¬ 
sider  time-varying  deterministic  linear  systems;  the  notion  of  a  determin¬ 
istic  observer  and  estimator  for  a  continuous  linear  system  is  defined  and 
we  prove  that  classes  of  observers  and  estimators  can  be  constructed  if 
the  dynamics  of  the  system  are  known.  Equivalent  classes  of  observers  and 
the  classes  of  minimal  order  observers  are  defined  and  some  preliminary 
results  on  parameter^ing  equivalent  classes  of  observers  are  obtained.  In 
section  4.3,  we  extend  the  deterministic  notions  to  stochastic  systems 
where  we  show  that  some  classes  of  observers  yield  unbiased  estimates.  By 
some  physical  considerations,  we  restrict  the  classes  of  observer-estimators 
that  shall  be  considered.  We  then  determine  tt.e  class  of  minimal  order 
observers  that  yield  minimum  vai lance  estimates  by  formulating  the  problem 
as  finding  the  minimal  function  of  a  certain  restricted  solution  set  and 
then  using  theorem  2.6.3.  We  then  show  that  the  class  of  minimal  order 
optimum  observer-estimators  yields  the  conditional  mean  estimates  of  the 
stage.  This  reveals  the  true"  nature  of  the  derived  minimal  order  optimum 
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observer-estimator  .  In  section  4.4,  the  notion  of  detectability  of  con¬ 
tinuous  linear  time  system  is  defined,  and  the  asymptotic  behavior  of  the 
optimum  estimator  is  studied  in  terms  of  detectability  and  observability 
of  the  system.  In  section  4.5,  we  have  some  general  discussions  on  the 
approaches,  results  and  further  applications.  In  section  4.6,  detailed 
literature  connected  with  the  development  in  this  chapter  is  listed. 

Conceptually,  there  is  little  difference  between  discrete  and  con¬ 
tinuous  time  linear  systems;  therefore  we  would  expect  the  results  obtained 
in  this  chapter  will  be  quite  similar  to  those  of  Chapter  III.  One  marked 
difference  between  the  discrete  and  continuous  time  cases  is  that  for  the 
discrete  time  case,  the  observation  statistic  is  sequential,  and  so  each 
bit  of  observation  conveys  finite  amount  of  information  in  an  accumulative 
manner;  whereas  in  the  continuous  time  case,  we  have  only  a  priori  infor- 
mation  before  any  observation  is  made,  and  when  an  observation  is  made  at  the 
initial  time  we  have  a  sudden  increase  of  information  within  a  very  small 
interval  of  time  due  to  some  noise-free  observatior  component.  We  would 
expect  this  l:jump"  in  information  to  be  reflected  in  the  initial  condition 
of  the  optimum  observer-estimator. 

4 . 2  Classes  of  Observers  for  Continuous  Linear  Systems 

In  this  section,  we  shall  consider  a  linear  time-varying  continuous 

Q 

system  described  by 

(state  eq.)  x(t)  =  A(t)x(t)  +  B(t)  u(t) 

£(t)  =  C(t)x(t) 


(output  eq.) 
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where  L  We  shall  assume  Chat  C(t)  is  a  differentiable  time- 

varying  nxm  matrix  of  rank  m,  for  all  t  £  £ »°°3  (n  -  m)  .  For  a  fixed 
t  c  £ tQ ,°°]  the  set  of  complementary  matrices  of  order  s  for  C(t)  is  denoted 
by  fi(C(t);m,s,n)  =  (T(t)  £  M  :  N(T(t))  nH(C(t))  =  0^  c  Rn).  We  note 
that  T(t)  £  P.(C(t)  ;m,ssn)  if  and  only  if  there  exist  matrices  P_(t) , 

V(t)  (of  appropriate  dimensions)  such  that 

P(t)T(t)  +  V(t)C(t)  =  I  .  (A. 2.1) 

—  —  —  —  — n 

Definition  4.2.1:  A  linear  time  varying  system  of  dimension  s  t  n  -  m 

.z(t)  =  F(t)z(t)  +  D(t)x(t)  +  G(t)u(t)  ;  z.(rQ)  =  (4.2.2) 

is  an  s-order  observer  for  the  system  §,  if  for  some  choices  of  ,  the 
solution,  z(t)  of  (4.2.2)  equals 

_z(t)  =  T(t)x(t)  ;  t  >  tQ  (4.2.3) 

for  some  T(t)  z  P(C(t) ;m,s,n) ,  t  >  t  .  We  shall  also  say  that  the  observer 
—  —  o 

is  described  by  T(t),  t  >  t  ;  and  refer  to  such  an  observer  by  the  symbol 

®r 

Let  T(t)  be  an  sxn  matrix  which  satisfies  the  differential  equation 
(t  >  tQ) 

T  (t)  =  F(t)T(t)  -  T(t)(A(t)  -  L(t)C(t)  +  D(t)C(t)  ;  T(t*)  =  (4.2.4) 

where  L(t),  _F(t),  i)(t),  are  some  prescribed  matrices  of  appropriate 
dimensions.  If  we  construct  a  time  varying  system  of  dimension  s  ^  n  -  m: 

(^:  1  (t)  =  F(t)z(t)  +  (D(t)  +  T(t)L(t))i(t)  +  T(t)B(t)u(t)  (4.2.5) 


tr.en  using  (4.2.-),  ve  have 

^  (T(t)x(t)  -  z(t))  =  F(t) (T(t)x(t)  -  z(t»  (4.2.6) 

and  if  ve  cheese  T  x(t  )  =  z(r*),  inen  7(t)x(t'  =  z(t),  t  •  t  . 

— o  —  o  —  o  __  o 

-  c 

Therefore  will  be  an  s-order  observer  for  §  ^  if  by  sore  appro¬ 
priate  choices  of  L(t),  F(t),  D(t),  the  solution,  T(t)  of  (4.2.4)  will 
be  in  the  set  of  complementary  matrices  of  order  s  for  C(t),  t  ■  t^. 

3v  assumption,  C(t)  is  differentiable  for  all  t  *:  it  ,*];  thus  there 

—  o 

exists  a  function  T(t)  •  ..(C(t)  ;m,s,r.) ,  t  •  t  ,  such  chat  T(t)  is 
—  —  o  — 

differentiable. 

Theorem  4.2.2:  Let  T(c)  .‘.(C(t)  ;m.s.n) .  t  ••  t  .  and  T(t)  is  differentiable 

—  —  Q  ~ 

in  the  interval  (t  ,3).  Then,  there  exists  a  class  of  s-crder  observers 
o 

v’nicii  are  all  described  bv  T(c),  t-c  for  the  system  . 

Proof:  Let  j?(t),  V(c),  t  •  £  ,  be  matrices  of  appropriate  dimension  such 
that 


P(t)T(t)  +  V(t)C(t)  =  I  ;  t  •  c  .  (4.2.7) 

—  _  _  _  — n  o 

Choose  for  t  >  t 

o 

D(t)  -  T(t)A(t)V(t)  +  T(t)V(t)  (4.2.8' 

F(t)  =  T(t)A(t)P(t)  -r  T(t)P(t)  (4.2.9) 

T  =  T(tL)  (4.2.10) 

— o  ~  o 

where 

A<t)  =  A(t)  -  L(t)C(t) 


(4.2.11) 
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an d  L(t)  is  an  arbitrary  nxm  matrix.  With  these  choices  of  J3(t),  jF(c)  ,  and 
T  ,  ve  have  the  solution  of  (4.2.4) 

T(t)  =  C„(t,C*)T(t*):=(t*  t)r!  C_(t,T)[T(T)A(T)+^Jf^-  V(T)C(T)3^(t,t)dT 
—  — r  o  —  o  — s\  o  ^  — t  —  —  Ci  —  —  —A 

°  (4.2.12) 

where  c„(t,t+)  and  c.(t,t~)  are  fundamental  matrices  associated  with  F(t) 

— r  o  — ,i  c 

and  A(t)  respectively.  Using  (4.2.7)  and  (4.2.9)  the  integrand  of  (4.2.12) 
becomes 


-c 

- 1 


—  '  L 

J._(t,:)li(:)A(:)  -  £(-)  C(r)]c- (' ,t)dr  =  '  ^(c,t)T(t) 


d- 


_-p.(t,  )  «.;(-.t)  -  -  i'(z)C£(z,t)]d:  t(c)  cf.(t,toJ_i_(to)j;^{r.ott) 

(4.2.13) 


Combining  (4.2.12.)  and  (4.2.13)  ve  have 


T(t)  =  T(t)  .'.(C(t)  ;n,s,n)  ,  t  >  t  .  (4.2.14) 

—  —  —  o 

So  an  s-order  observer  can  be  constructed  by  (4.2.5).  We  note  that  by 

choosing  different  L(t)  £  >!  ,  t  >  t  ,  we  obtain  a  class  of  observers  de- 

-  nm  o 

scribed  by  the  same  T(t),  t  >  t  .  For  a  fixed  T(t)  £  2(C(t) ;m,s,n) , 

—  o  —  — 

lc 

t  *  t  ,  and  a  fixed  L(t)  £  M  ,  t  >  t  ,  we  shall  use  the  symbol  0”  (L)  to 
o  —  nm  o  -X 

represent  the  observer  which  is  specified  by  T(t)  and  L(t) ,  and 

1  c  1  o  ** 

(L)/L(t)  £  >5  }  the  class  of  observers  which  is  specified  bv  T(t). 

i  1  —  nm  — 

lc  lc  lc 

For  each  (L)  G  ,  we  shall  associate  with  it  an  estimators*  (L) 

described  by  (Figure  4.1) 

z(t)  =  (T(t)(A(t)-L(t)C(t))P(t)+i(t)P(t))z(t)+(T(t)(A{t)-L(t)C(t)V(t) 

p^c (i  \  . 

T  ^  +T(t)V(t)+T(t)L(t))x(t)+|(t)B(t)u(t) 

w(t)  =  P(t)z(t)+V(t)yU) 


(4.2.15) 


CONTINUOUS  TIME  LINEAR  SYSTEM 


(A. 2. 16) 


If  we  know  x(t+),  Chen  bv  setting 
—  o 

z(t+)  =  T(t‘l)x(tf) 

—  o  -  o  —  o 

we  have  from  (A. 2. 6)  and  (A. 2. 16)  that 

w(t)  =  P(t)T(t)x(t)  +  V(t)C(t)x(t)  =  x(t)  .  (A. 2. 17) 

*| 

But  usually  x(tJ)  is  unknown,  and  so  if  we  want  to  useS^  (L)  as  an  esti- 
—  o  i 

mating  device,  we  would  restrict  the  initial  condition  of  _z(c~)  to  be  in 
the  range  space  of  T(tJ). 

Let  V(t)  t  M  ,  t  >  t  ,  be  a  fixed  differentiable  matrix,  Associated 
nm  o 

with  it  is  a  set  of  matrix  functions  o'!  =  {T(t)  e  M  ,  t  >  t  /T(t)  is  dif- 

V  —  sn  o  — 

ferentiable  on  (to,*)  and  P.(t)T(t)  +  ^(t)C(t)  =  f°r  some 

P(t)  e  M  ,  t  e  (t  ,<»);s  in-  ml.  For  a  fixed  T(t)  eof,  we  can  associate 
—  ns  v  o  '  J  —  V 

1  c 

with  it  a  class  of  observers  and  a  class  of  estimators 

S„lc  =  {G^C(L)/L(t)  z  M  }.  Therefore,  for  a  fixed  V(t),  we  can  associate 
i  T  -  nm  - 

lC  r»  C 

with  it  different  classes  of  observers  <3^  .  T(t)  z  <1^,  of  different  orders. 

For  a  fixed  V(t),  t  >  t  suppose  that  (I  -  V(t)C(t))  has  rank 
—  o  — n  - 

n  -  p ,  (p  1  m)  :  then  the  class  ~^C(L)  =  (<3^C(L) /T(t)  and  T(t)  e  ^n(n 

has  full  rank,  t  e  (t  j*)}  is  called  the  class  of  minimal  order  observers 

o 

associated  with  V(t)  and  parameterized  by  L(t).  We  can  define  the  notion 
of  equivalent  representation  as  in  the  discrete  case  (Definition  3.2.5). 


For  the  rest  of  this  chapter,  V(t)  is  always  assumed  to  be  differentiable 
on  t  e  (t  ,°°] . 


h 


1 
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r.a 


'r  i  :  .xvu  t  * ,  t 


such  that  T  -  (t)C(l)  hat.  ran-. 


.1  -  ?,  t  t^,  It-c  C-^''(L),  i(t)  --  be  a  given  observer  of  order 

lc  ^  c 

s  2  n  -  p.  Then  there  exists  a  n  -  p  order  observer  G^  (L) ,  T(t)  eo  such 
lc  lc 

that  (L) ,  G,j.  (L)  are  equivalent, 
lc 

Proof:  Let  Gj  (L)  be  a  given  s-order  observer;  its  associated  estimator 

lc 

£^.  (L)  is  described  by 


z(t)  =  (T(c)(A(t)-L(t)C(t)P(t)+T(t)P(t))z(t)+(T(t)(A(t)-L(c)C(t) 

lc 

eT  (L):  V(t)+T(t)V(t)v(t)+T(t)B(t)u(t)  ;  z(t*)eS  =  {T(t+)a' 'ieRn} 

w(t)  =  P(t)z^(t)+V(t/v(t) 

(4.2.18) 

with  P(t),  T(t)  satisfying  (4.2.1)  and  jz(t)  £  RS,  s  £  n  -  p.  Since 
-  V(t)C(t)  has  rank  n  -  p,  we  may  assume  without  loss  of  generality 
that  P(t)  is  of  rank  n  -  p,  P(t)  is  a  time  varying  linear  transformation 

s  g 

from  R  ■*  Rn.  We  can  break  the  transformation  into  two  steps:  map  R  to 
Rn  ^  by  a  tire  invariant  transformation  K,  then  from  Rn  p  to  R”  by  an  appro¬ 
priate  time  varying  transformation  £(t)  i.e., 


P(t)  =  P ( t ) K 


P(t) 


'^(n-p)  —  *  *(n~p)s 


(4.2.19) 


1  C  v 

Let  us  construct  an  n  -  p  order  observer  G^  (L)  with  T(l)  =  K  T(t)  and 

•f*  -  -{•  |] 

the  restricted  observer's  state  initial  condition  jj(tQ)  e  S  =  {K  T(t_)u*u  >:  R 
First  we  see  from  (4.2.19)  that 


o - 


P(t)T(t)+V(t)C(t)  =  P(t)K  T(t)+V(t)C(t)  =  P(t)T(t)+V(t)C(t)  =  1 


(4.2.20) 


thus  we  conclude  that  T(t)  e  a 


*T  C 


lc, 


Let£^,  (L)  be  the  estimator  associated 


with  G^(L).  To  prove  the  lemma,  we  need  to  verify  that  w(t)  -  w';)  for 
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_  1c 

all  possible  u(t)  and  x(t)  where  w(c)  is  Che  output  of  (L)  .  Let 

z(tH)  =  T(t+)o  for  some  u  e  Rn  and  pick  z(t*)  =  K  T(t*)a;  then  one  can 
—  o  —  o  ■ —  o  o 

easily  show  that  by  construction 


z(t)  =  K  z(t)  ;  t  »  t 


(4.2.21) 


for  all  x(t)  and  u(t) .  Then  we  have 


w(t)  =  P(t)z(t)+V(t)x(t)  =  P(t)K  z(t)+V(t)x(t)  =  P(t)z(t)+V(t)x(t)  »  w(t)  . 

(4.2.22) 

Conversely,  if  z(t*)  =  K  T(tQ)£,  pick  2^CQ)  =  T(to)ci;then  we  have  (4.2.21) 

and  (4.2.22)  in  the  same  manner,  and  the  lemma  is  proved. 

Theorem  4.2.4:  Let  V(t)  e  M  ,  t  >  t  ,  such  that  the  rank  of  V(t)  and 
- -  nm  o 

I  -  V(t)C(t)  are  p  and  n  -  p,  respectively.  For  a  fixed  L(t)  t  M  the 
— ■ n  —  nm 

class  of  observer;  &^C(L),  T(t)  e  T^  are  all  equivalent. 

Proof:  Let  P(t)  e  M  ,  , ,  T(t)  e  M,  ,  such  that 

-  -  n(n-p)  —  (n-p)n 


P(t)T(t)  +  V(t)C(t)  =  [P(t)  .*  V(t))  ...  =  I  .  (4.2.23) 

C(t) 


Denote  the  column  vectors  of  V(t)  bv  v. (t) ,  i  =  1,  ...,  m: 

—  •  — i 


V(t)  -^(t)  v2(t)  ... 


(4.2.24) 


Since  V(t)  has  rank  p,  {v^  ^)(t)}?_^  form  an  independent  set,  where  a^(’) 
is  a  permutation  of  1,  ...,  m  and  v^  >  p  are  dependent  on  {v^  (i)  -'i=i  ■ 

Rearranging, if  necessary,  we  may  assume  for  a  fixed  t: 


v(t)  =  rv^O  :  v2(t)] 


(4.2.25) 
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with  V.  (t)  t  M  and  of  rank  p,  while 
— 1  np 

V,(t)  =  Vx(t)M(t)  ;  M(t)  s  Mpfc_p)  .  (4.2.261 

The  matrix  C(t)  is  also  rearranged  accordingly  (if  necessary);  we  may  assume 

cx(t) 

*  •  • 
c2(t) 

Since  £(t)  is  of  full  rank,  (4.2.23)  implies  that 

C2(t)  »  S(c)l(c)  ;  N(t)  s  **(n-p) (n-p)  '  <4'2-2S> 

Using  (4.2.23)  to  (4.2.23),  we  have  for  a  fixed  t: 

’l(t) 

lP(t):v.(t)]  ...  =1  (4.2.29) 

’  1  C-^t)  +  M(t)N(t)T(t)  ~n 

Since  (P(t)  V  (t)]  z  M  ,  (4.2.29)  implies  that 
—  .  — i  nn 

T(t)P(t)  -  ;  T(t)vl(t)  =  0(n_p)p  ;  J 

+  M(t)S(t)T(OV1(c)  -  CjCc^Ct)  -  Ip  .  1(4.2.30) 

From  (4.2.25),  (4.2.26)  and  (4.2.30),  we  have 

T(t)P(t)  =  I  ;  T(t)V(t)  =  0,  .  (4.2.31) 

_  --  — n-p  —  — (,n-p)m 

We  note  that  under  the  assumption  on  V(t),  (4.2.31)  is  true  for  all 

t  '  t  . 
o 

From  lemma  4.2.3,  we  see  that  to  ^rove  the  theorem  we  need  only  to 

1  c  pc 

prove  that  all  observers  03^  (L)  e  (L)  are  equivalent. 

let  03^C(L)  c  ir^c(L)  be  arbitrary,  i  =  1,  2.  The  associated  estimators 
i 

are  described  by 


C.  (t)  t  M 
— 1  pn 


C.(t)  e  M,  - 
—2  (xa-p)n 


(4.2.27) 
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z.  (t)  =  (T.  (t)  (A(t)-L(t)C(t))P.  (t)+T.  (t)P.  (t))z.  (t)+(T.  (-)L(t)+T.  (t)l(t) 

-i  — 1  —  —  —  -I  — l  — l  — l  — l  —  —i  — 

•fT.  (t)  (A(t)-L(t)C(t))V(t$jflt)-hT-  (t)B(t)u(t) 

— i  —  —  —  —  —a  — 

w.  (c)  -  P.  (t)z.  (t)-fV(t)y(t)  ;  z.(t+)tS  =  {T.  (t+)a!acRn} 

— 1  — x  — 1  —  — 1  o  i  — i  o  — — 


(4.2.32) 


Without  loss  of  generality,  we  may  assume  that  the  P.(t)  t  M 


n(n-p) 


are  of  rank  n  -  p.  Then  there  exists  a  nonsingular  matrix  K(t)  t  M, 
such  that 

-1. 


(n-p) (m-p) ’ 


Px(t)  =  P2(t)K(t)  ;  P,(t)  =  Px(t)K  "(t)  (4.2.33) 


and  so  we  also  have 


-1, 


K(t)T1(t)  =  T2(t)  ;  T^(t)  =  K  (t)T„(t)  .  (4.2 


.34) 


Let  us  define 


jKt)  =  K(t)z_x(t)  .  (4.2.35) 

Using  (4.2.32)  to  (4.2.35),  we  obtain  the  equation  for  z(t); 

|(t)  =  (T2(t)(A(t)-L(t)C(t))P(t)+K(t)T1(t)P2(t)+K(t)K'1(t))z(t) 
4(T2(t)L(t)+K(t)T1(t)V(t)+T2(t)(A(t)-L(t)C(t))V(t))x(t) 

z(t^)  =  K(t*)T-  (t+)a  =  T.(t+)a  s  S,  (4.2.36) 

since  J\(t),  T\(t)  satisfy  (4.2.23),  i  =  1,  2;  thus,  by  (4.2.31),  (4.2.33) 
and  (4.2.34)  we  can  easily  show  that 

K(t)T1(t)P2(r)  +K(t)K_1(t)  =  i2(OP2(t) 


(4.2.37) 
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K(t)T1(t)V(t)  =  K(t)T,(c)V(t)  (4.2.38) 

Substituting  (4.2.37),  (4.2.38)  into  (4.2.36)  and  comparing  with  (4.2.32), 

we  see  for  any  given  z:(t*),  we  can  pick  an  appropriate 

z9(t  )  •:  S9(z(t  )  =  z0(t  ))  such  that 
—L  o  2  -  o  —2  o 

z(t)  =  z,(t)  (4.2.39) 

and  so 


w(t)  =  P^Oz^t)  +  V(t)^(t)  =  P2(t)K(t)z1(t)  +  V(t)£(t) 

=  P9(t)^,(t)  +  V(t)^(t)  =  w,(t)  .  (4.2.40) 

lc  ic 

Therefore  &  (L)  is  an  equivalent  representation  of  gZ^L) ;  similarly,  we 

1  .  2 

can  prove  £;J;C(L)  is  an  equivalent  representation  of  £lc(L)  and  the  theorem 

1  T2 

follows. 

Note  that  the  results  are  different  from  those  in  the  discrete  case. 

We  see  that  only  in  some  special  cases  equivalent  classes  of  observers 
are  parameterized  by  V(t)  z  M  and  L(t)  z  M  .  Because  of  this  difference, 
our  approach  to  the  problem  of  designing  "nice-behaved"  observers  and 
associated  estimators  for  the  continuous  system  will  be  slightly  different 
from  that  used  in  the  discrete  case. 

4 . 3  Optimum  Class  of  Observers  for  Linear  Stochastic  Systems 

Consider  a  stochastic  system  described  by:  (Figure  4.2) 


x(t)  =  A(t)x(t)  +  B(t)u(t)  +  C.(t) 


"C  (t)x(t)  +  n(t) 

l(t)  = 

C2(t)x(t) 


(4.3.1) 


TWt  STRUCTURE  OF  A  CONTINUOUS  TIME  STOCHASTIC 
LINEAR  SYSTEM  SS  WITH  SOME  OBSERVATION 
CHANNELS  NOISE-FREE 
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m 

where  >c ( t)  e  Ru,  u(t)  e  Rr,  4.(t)  e  Rn,  n(t)  e  R  y/t)  e  Rm>  (m  -  • 

We  assume  that  x.(to),  {£(t),  t  ^  to},  (n.(t) ,  t  £  tQ)  are  independent 

statistics.  x(t  )  is  a  Gaussian  random  vector  with  mean  x  and  covariance 

—  o  — c 

E  ;  C.(t),  _n(t),  t  £  t  are  white  Gaussian  noises  with  properties 

R(t)dt  ;  t2  >  t 

e| f  2  n(t)dt j  =  0  ;  E j (f  2  n(t)dt)(  f  2  n(t)dt  )  j =  2  £(t)dt  ;  t2  >  t± 

'  C1  C1  C1  tl 

(4.3.2) 

where  R(t)  -  0,  R(t)  e  M  and  Q(t)  >  0,  Q(t)  c  M  .  The  control  u(t)  is 
—  —  —  nn  —  m^m^  - 

known  function  of  time. 

Let  us  denote  the  noisy  observation  by  y^(t)  and  the  noise-free  obser¬ 
vation  by  y.2(t) : 

y^vt)  =  G1(t)x(t''  +  _n  (t)  ;  y2(t)  = 

Our  objective  is  to  find  a  "filter"  whose  output  will  be  an  unbiased 
minimum  mean  square  estimates  of  2i(t)  .  Since  x.(t)  is  a  Gaussian  random 
process  (see  Chapter  2,  section  2.3),  we  may  restrict  ourselves  to  consider 
only  linear  filters  [47].  Thus  we  may  assume  that  the  estimate  of  x.(t)  is 
given  by 

t 

£:  w(t)  =  H(t,x)y(-)dT  +  V(t)y_(t)  (4.3.4) 

t 

o 

where  H(*,*)  is  an  nxm  matrix  whose  elements  are  differentiable  in  both 
arguments.  If  we  demand  the  system  £  to  yield  unbiased  estimates  of  x(t), 
then  £  can  be  realized  by  an  s-order  observer  &2c (L) '  and  its  associated 

The  superscript  2  is  to  indicate  that  system  §2  is  considered. 


E jf  2  I(t)dtJ  =  0  ;  eJ(|  2  S(t)dt)(j  2  C(t)dt)  j  =  2 

**  -t  -i 


estimator  E“''(L)  (see  section  4.5).  For  this  reason,  we  may  view  observer- 
estimators  as  estimating  devices. 

Let  us  restrict  ourselves  only  to  some  special  classes  of  observers. 
First  we  note  that  in  the  discrete  analog,  the  optimum  observers  are 
compatible  with  respect  to  the  noise-free  observation.  This  is  one  in¬ 
trinsic  property  of  the  optimum  observers,  and  this  property  should  be 
preserved  when  we  pass  from  the  discrete  to  the  continuous.  Thus,  we  s^all 
only  consider  observers  which  are  compatible1  with  respect  to  the  noise- 
free  observation  _y2(t).  Since  y^(t)  contains  white  noise  in  the  measure¬ 
ment,  therefore,  in  order  to  obtain  reasonable  estimate,  we  shall  not ’pass" 

jr. (t)  without  filtering.  These  physical  consideration  allow  us  to  consider 

2c 

only  those  observer  6^  (L)  which  are  compatible  and  parameterized  by 

L(t)  e  M  arbitrary  and  V(t)  of  the  form 
~  nm  — 

V(t)  =  (0  !  V2(t)]  V2(t)  £  Mn(m_m  )  .  (4.3.4) 

All  such  V(t)  are  of  rank  i  m  -  m^  =  m2. 

Theorem  4.3.1:  Let  V(t)  be  of  the  form  (4.3.4)*  if  there  exists  an  observer 
2  c  c 

0,j,  (L) ,  T(t)  e  3"v,  which  is  compatible,  then  rank  V(t)  =  m2  and  rank 

(I  ~  V(t)C(t))  =  n  -  in  . 

— n  “  —  l 

lc  lc 

Proof :  By  lemma  4.2.3,  we  may  assume  that  there  exists  (L)  £  (L) 

2c 

which  is  compatible.  Let  (L)  be  the  associated  estimator  and  w(t)  the 
resulting  estimate.  Using  (4.3.4),  we  have 

e(t)  =  w(t)-x(t)  =  P(t)^(t)+V(t)y(t)-x(t) 

=  P(t)£(t)+V2(t)jC2(c)2i(t:)''2i(t:)  =  P.(t)  (z(t)-T(t)x(t))  .  (4.3.5) 

1  Compatibility  is  defined  as  in  discrete  case. 


See  section  3.4. 
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By  compatibility  we  must  have 

C,(t)e(t)  =  C9(c)?(t)(z(t)  -  T(t)x(t)>  =0  a.s.  (4.3.6) 

thus  in  particular  if  j\(t)  i  0,  r^(t)  l  0, (4.3.6)  implies  that 

C_(t)P(tVi.„v.t,t+)T(t+)a  =  0  ;  a  e  Rn  arbitrary  (4.3.7) 

_/  _  — 2  e  —  o  ~  -  — 

where  1(c)  is  give  a  by  (4.2.9).  Since  6“C(I.)  t  ~£C(L),  therefore  T(t+)  mav 
—  iv  —  c 

be  assumed  to  be  of  full  rank,  and  so  (4.3.7)  implies  that 

C,(t)P(t)  -  0  .  (4.3.8) 

Using  (4.3.5),  we  have 

C9(t)e(t)  =  C2(t)V2(t)C9(t)x(t)  -  C2(t)x(t)  =0  a.s.  (4.3.9) 

x(t)  can  be  an  arbitrary  vector  in  R°;  so  we  conclude  that 

C9(t)V_(t)  =  I  (4.3.10) 

— 2  —Z  ~m2 

and  that  rank  V^(t)  =  m^. 

From  (4.3.8),  we  have 

C9(t)P(t)T(t)  =  0  ;  rank  P(t)T(t)  i  n  -  m0  (4"  3.11) 

P(t),  I(t)  satisfy 

P(t)T(t)  +  V9(t)C9(t)  =  I  .  (4.3.12) 

Equations  (4.3.12)  and  (4.3.10)  imply  rank  £(t)T(t)  in-  ra2.  Together 
with  (4.3.11)  we  have 

rank  (i^  -  V^t)C(t))  =  rank  £(t)T(t)  ~  n  -  m9 


(4.3.13) 
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By  theorems  4-2.4  and  4.3.1,  we  see  chat  ail  V(t)  of  the  torm  ii.i.4^ 

can  be  classified  into  two  classes:  either  all  G_'fL),  T/t)  o^T  are  all 

uncompatible  or  all  6“  (L) ,  T(t)  t  oy  are  all  compatible  and  equivalent. 

We  nay  call  the  former  class  of  V(t)  uncompatible  and  the  latter  class  of 

V(t)  compatible.  Thus, the  classes  of  observers  which  we  shall  consider  are 

parameterized  by  I-(t)  t  M  and  compatible  V(t)  of  the  form  (4.3.4). 

_ run  - 

Let  V(t)  be  a  fixed,  differentiable  matrix  function  of  the  form  (4.3.4) 
which  is  compatible;  and  let  L(t)  e  be  arbitrary.  From  theorem  4.2.4, 
all  observers  fc^c(L) ,  T(t)  e  are  equivalent  and  thus  yield  the  same  error 
dynamics.  Let  (~“C(L)  t  its  associated  estimator  £Z°(L)  is 


described  by: 


e;c(D: 


(T(t)(A(£)-L(t)C(t))?(tKT(t)P(tWt)+(T(t)(A(t)-L(t)C(t))V2(t) 

+T(t)V0(c)+T(t)L2(t))x0(c)^T(c)L1(c)v1(t)-i-T(t)B(c)u(c) 

P(t)z(t)+V,(t)v9(t) 


(4.3.14) 


L(t)  =  [Lx(c)  :  L2(t)]  ;  Lx(t)  £  M  ,  L.,(t)  e  (4.3.15) 

P(t)  e  M  ,  .,  T(t)  e  il,  .  satisfy  (4.3.12)  and,  in  addition,  they 

-  n(n-m2)  —  (n-m^)n 

satisfy: 


T(t)V  (t)  =  0  ;  C,(t)P(t)=0  ;  C (t)V (t)  -  I  ; 

—  —2  ~nia2  ~2  —  — — Z  — Z 

T(t)P(t)  =  I 
—  “ 


(4.3.16) 


We  can  simplify  the  structure  of  £^,  (L)  by  using  (4.3.15)  and  (4.3.16): 
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T(c)Lu)CU)?U)  =  KcH^CtjC^O+L^CtJC^tcJjPCt)  =  KOL^OC^OPCt) 

(4.3.17) 

T(c)L,(t)  -  T(t)L(t)C(c)V,(t)  =  -KOLjttJC^OV^t)  (4.3.18) 

Substituting  (4.3.17),  (4.3.18)  into  (4.3.14),  the  structure  of  the  estimator 
2^C(L)  is  given  by  (Figure  4.3) 

1(0  =  (T(t)A(t)P(t)-T(t)P(t)-T(t)L1  (tK  (t)P(t|z(t)+T(t)B(t)u(t) 

C“C(L)* 

T  >T(t)L1(t)il(t)+(T(t)A(t)V7(t)+T(t)V,(t)-T(t)L1(t)C1(t)V2(t))y2(t) 

«(t)  =  P(t)z^(t)-rV7(t)x7(t) 

(4.3.19) 

■>C 

By  demanding  (L)  to  give  unbiased  escinates  of  x(t) ,  we  set  (see  also 
section  4.5) 

z(t  )  =  T(t  )x  (4.3.20) 

—  o  —  o  — 0 

where  T(t  ),  ?(t  '  satisfy 
—  o—o 

P(t  )T(t  )  -r  V_ (t  )C,(t  )  =  1  .  (4.3.21) 

—  o—o  —  2  o  — 2  o  — n 

Using  (4.3.1),  (4.3.2),  (4.3.16),  (4.3.19)  and  (4.3.20),  we  have  the 
dynamics  of  the  error  process,  e(t)  k  w(t)  -  x(t),  given  by:  (see  Appendix  C) 

e(t)  =  (A(t)-y2(t)C2(c)-P(t)T(t)L1(t)C1(t))e(t)+(V2(L;C2(t)-In)i(t) 

+P(t)T(t)L,(t)^(t)  (4.3.22) 

a 

e(t  )  =  ( I  -  V ■  (t  )C.(t  ))(:<  -  x(t  )) 

—  o  — n  — 2  c  — 2  o  ~o  —  o 

and 

c2(t)  k  C2(t )  4  C7(t)A(t) 


(4.3.23) 


UNBIASED  MINIMAL-ORDER  OBSERVER -ESTIMATOR 


\ 


Define 


L.(t)  £  P(t)T(t)L. (t)  e  >1 
— 1  —  —  — 1  nm. 


(4.3.24) 


The  error  covariance  is  given  by  (see  chapter  2,  section  2.3) 

|(t)  =  [A(t)-V,(t)C2(t)-Li(t)C1(t)3Z(t)+Z(t)[A(  -V9(t)C,(t)-L1(t)C1(t)]' 

+  [ln-V2(t)_C2(t)  jR(t)  [In-V,(t)C2(t)  ]  ,+L1(t)S(t)L^(t) 


z(t  )  =  (I  -  V  (t  )C(t  ))£  (I  -  v  (t  )C.(t  ))’ 
—  o  — n  2  o  —  o  - o  - n  2  o  2  o 


(4.3.25) 


We  note  that  the  dynamics  of  the  error  covariance  are  dependent  on  V^t), 

t  £  [t  ,*■]  and  L,(t),  t  £  [t  ,®].  Note  that  V  (t)  and  L.  (t)  are  not  arbi- 
c  — l  o  2  - l 

trary  but  V2(t)  has  to  satisfy  (4.3.10)  and  L^(t)  is  related  to  L^(t), 
which  is  an  arbitrary  matrix,  by  equation  (4.3.24).  To  find  the  optimum 
class  of  observers,  we  are  to  find  a  pair  {V  (t)»  Lj(t)},  which  may  be  a 
nonunique  pair,  with  the  above  constraints  which  will  give  the  "least" 

,  *  -  "k 

nonnegative  definite  covariance  matrix.  Each  such  pair  {V7(t),  L^(t)r 

specifies  an  optimum  class  of  observers.  When  vtf  (t)  ,  L^(t)}  ranges  over 

ail  constrained  pairs,  we  generate  the  solution  set,  B  ,  of  (4.3.25).  The 

*  ° 

minimization  problem  is  equivalent  to  finding  {V2(t),  L*(t)}  which  will  yield 

the  minimal  function  with  respect  to  the  solution  set  R^  . 

o 

Theorem  4.3.2:  Let  (t)R(t)£2  (t)  -•  Ch,  then  there  exists  a  unique  con- 

•Jj  ^ 

strained  pair  {_V  (t),  t  c  [ t q « 3 ,  L^(t),  t  e  [ Cq ,°° ] }  which  yield  the  unique 

*  -  "k  „  -k 

minimal  function.  Z_  (t),  with  respect  to  .  V.2(t),  Lj(t)  are  given  by 


.  (  Z  CMt  )(C.(t  )Z  Cl(t  ))  " 

*  ^  ^  j  _  )  —o—2  o  — 2  o  -o-2  o 

2  1  (Z*(t)Cl(t)  +  R(t)C:(t))A_1(t 


(4.3.26) 
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L*(t)  =  Z*(t)Cj(t)0  1(c) 


t  -  t 


(4.3.27) 


where 


£(t)  4  c2(t)R(t)Cj(t)  >  0 


(4.3.28) 


Z_  (t)  is  the  minimal  function  with  respect  to  and  is  given  by 

o 


-1 


* 


Z  (t)  =  (A(t)-R(t)C^(t)4  "(t)C2(t))Z  (t)+Z  (t)(A(t)-R(t)C'(t)l  "(t)C,(t))' 
-Z*(t)(C’(t)4_1(t)C2(t)+C[(t)2'1(t)C1(t))Z*(t)+R(t) 
-R(t)C;(t)A_1(t)C2(t)R(t) 


z“(t  )  =  :  -  z  c;(t  )(c.(t  )zcl(t  ))_ic_(t  )Z 

—  o  o  —cr~2  o  ~2  o  ~a~2  o  —2  o  ~o 

(4.3.29) 

Let  be  the  solution  set  of  (4.3.25)  when  (V2(t),  L^(t)}  ranges  all 
°  * 

possible  pairs;  £  (t)  is  also  the  minimal  function  with  respect  to  fit  ; 

-  o~ 

thus  Z_  (t)  is  the  Riccati  function  (see  definition  2.6.4). 

Proof:  Let  P^.  be  the  solution  set  of  (4.3.25)  when  { c )  >  L  ^  ( t )  }  ranges 
o 

all  possible  pairs.  Compare  (4.3.25)  and  (2.6.1)  with 

L  (o  —  — v^t,c0 (c0)) 


[^(0  :  v2(t)  ]  -  v(t) 


c1( t)' 


Dx(t) 


(4.3.30) 


'fi(t)  :  0 

o  •  o 


£(t) 


(t) 


L  --9  v  'J 


p,(o 
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Since  by  assumption  £9(t)R(t)C2(t)  >  (),  then  we  have 

■o(t)  :  oi  r  o  i  fst(t)  :  o 

...  '...  +  ...  R(t)(0  :  Cl(t)J  =  ...  *  ...  >0  (4.3.31 

.  o  :  o  J  l_c2(t)J  *  2  L  o  :  c2(or(Ocj(o. 

and  so  the  unique  minimal  function  is  given  by:  (see  (2.6.19)  to  (2.6.21)) 

|*(t)  -  (A(t)-R(t)C2(t)^-1(t)jC„  (t))Z*(t)-K*(t)  (A(t)-R(t)C^(t)A^(t)C2(t))  ’ 
-Z*(t)(C2(t)A"1(t)C2(t)+C|(t)fi~1(t)C1(t))Z*(t)+R(t)-R(t)C^(t)A"1(t)C2(c:)R(t) 


r*(t  )  =  z  •ui(t)(c,(t)u;(t))'1c,(t)E 

—  o  — o  — o — Z  o  — Z  o  — o — Z  o  — L  o  — c 


and  [L^(t)  I  _V  2  ( t )  ]  is  giver,  by:  (see  2.6.18) 


(4.3.32) 


*  *  *  f  fi  :  £  ] 

[l,(o  :  v  (t) ]  =  z  (t)(c;(t)  :  c,(t)]+(o  :  R(oc;(t)i  ...  •  ... 

1  •  2  -  -i  .  -2  -  •  -  2  to  :  arHt). 


t  >  t 


v*  (  )  -  :  c’(t  )(c,(t  )z  cl(t  ))' 

—2  o  — o~2  o  — 2  o  — o~*2  o 


(4.3.33) 


To  complete  the  prove  of  the  theorem,  we  need  only  to  show  that  V„(t) 

.  it  * 

satisfies  (4.3.10),  and  L,  (t)  is  related  to  some  L,  (t)  t:  M  via  (4.3.24) 

— l  - 1  nm^ 

From  (4.3.32),  we  see  that 


C„(t  )Z*(t  )  =  0  .'  (4.3.34) 

- 2  o  —  o  - 

Using  (4.3.32),  (4.3.28)  and  (4.3.23)  we  have 

^  (C2(t)I*(t))  =  C2(tK*(t)[(A(t)-R(t)C^(tU"1(t)C2(t)),-(C2(t)A"1(t)C2^t) 
+Cj(t)(T1(t)C1(t))Z*(t)] 


(4.3.35) 
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Thus  we  conclude  that 


JL 

C2(t)z"(t)  =  0  (4.3.36) 

Therefore  we  have 

C2(t)vJ(t>  =  [C2(t)Z*(t)C’(t)  +  A(t)]4-1(t)  =  1^  .  (4.3.37) 

Let 

L*(t)  =  Z.*(t)C|(t)^_1(t)  =  L*(t)  .  (4.3.38) 

We  can  easily  see  from  (4.3.30)  chat 

T(t)P(t)Lj(t)  =  (IR-V2(t)C2(c))l'(t)C{(t)^"1(t)  =  Z*(t)c;(t)^-1(t) 

-  & 

=  ix(t)  .  (4.3.39) 

* 

Thus  I  (t),  given  by  (4.3.31),  is  also  the  minimal  function  with  respect  to 

*t  * 
o 

We  now  have  the  structure  of  a  class  of  minimal  order  optimum  observers, 

2c  *  c  * 

°T  ^  e  'JV*’  ^  =  t--  ■  —2  (*■)})  an<^  ^ts  associated  estimators 

2c  *  r 

CT  (L  ),  T(t)  t  Tj*: 

(T(t)A(t)P(t)+T(t)P(t)-T(t)L*(t)C1(t)P(t))z*(t)+T(t)L*(t)x1(t) 
+(T(t)A(t)V2(t)+T(t)vJ(t)-T(t)L*(t)C1(t)V*(t)22(t) 

+T(t)B(t)u(t) 

w  (t)  =  P(t)_z  (t)  +  V„(t)£  (t)  ;  z*(t  =  T(t  )x  (4.3.40) 

JL  Z  —  O  —  O  ~ O 

*  * 

with  V2(t),  Lx(t)  given  by  (4.3.32),  (4.3.33)  and  (4.3.37);  P(t) .  T(t) 


2X(t)  = 


2C  y  ^ 

Ct  (I.  ) 


satis  fy 
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I(t)V  (t)  =  0  ;  C  (t)P(t)  =  0  ,  ;  T(t)P(t)  =  I  ;  c  i  t 

“  Z.  i»2n  o 

(4.3.41) 

(^(t)  is  continuous  t  e  [t  ,"],  thus  we  can  choose  P.t)  which  is  continuous 

for  all  t  z  [t  ,”].  From  theorem  4.3.2,  V.(t)  is  discontinuous  at  t  =  t  , 
o  —2  o 

and  so  from  (4.3.41),  T(t)  is  discontinuous  at  tQ.  Since  e  (t)  k  w  (t)  -  x(t) 

is  continuous  at  t  =  t  ,  we  have 

o 

e*(t  )  »  P(t  )(z*(t  )-T(t  )x(t  ))  =  P(tJ(z*(t+)-T(t+)x(t  ))  (4.3.42) 

U  O  O  C  O  **“  o  —  o  —  o  —  o 

and  using  (4.3.40)  and  the  fact  that  z  (t  )  =  T(t  )x  ,  we  have 

—  o  —  o  — o 


z\t+)  =  T(t  )x  -  T(t  )V*(tV(t  ) 

o  —  o  — o  —  o  — z  o  z  o 


(4.3.43) 


We  see  that  z_  {t)  is  discontinuous  at  t  =  tQ,  and  consists  of  the  a  priori 
guess  (1(^)2^)  and  a  correction  term  due  to  perfect  observation 
(l(t0)V2(^)22(tH0»-  The  detail  structure  of  t^C(L*),  T^t)  z  is  shown 

in  Figure  4.4.  What  we  have  obtained  is  a  class  of  optimum  mean  square 
estimators  among  a  restricted  class  of  estimators  being  considered.  For 
example,  we  have  not  considered  the  class  of  nonlinear  estimators.  Now  to 
prove  the  derived  minimum  order  optimum  observer-estimator  is  the  truly 
optimum  estimator,  we  appeal  to  the  projection  theorem.  It  is  clear  that  w  (t) 
is  a  linear  functional  of  £2(s),  s  €  [cq , t ]  and  ^(s),  s  e  [t  ,t],  we  shall 
prove  chat  the  error  process,  e*(t)  k  /(t)  -  x(t) ,  satisfies  the  projection 
equations 


E{e  (t)^(s)}  =0  ,  s  e  [tQ,t)  ;  E{e*(t)^(t)}  =0  , 


s  e  Uo,t] 


(4.3.44) 


This  implies  that  (see  discrete  analog  and  Appendix  B)  the  optimum  class 

of  observers  will  yield  (a.s.)  the  conditional  mean  estimates  of  _x(t),  and 

2c  c 

thus  reveals  the  truly  optimum  nature  of  (L  ),  T(t)  e 
2c  *  c 

By  using  £T  (L  ) ,  T(t)  e  3^*,  as  an  estimating  device,  the  corre¬ 
sponding  error  process  will  satisfy 


e*(t)  -  (A(t)-V*(t)C2(t)-L*(t)C1(t))e*(t)+(V*(t)C2(t)-In)C(t)+L*(t)n(t) 


c*(t  )  «  (I  -  V*(t  )C9(t  ))(x  -  x(t  )) 

—  o  — n  — 2  o  — 2  o  — o  —  o 


(4.3.45) 

where  V2(t),  L^(t)  are  given  by  (4.3.32),  (4.3.33)  and  (4.3.37). 

Lemma  4.3.3:  Let  {e  (t) ,  t  2  t  }  be  a  random  process  satisfying  (4.3.45); 

-  —  Q 

and  x(t)»  t  -  tQ,  be  described  by  (4.3.1)  with  u/t)  =  (),  t  -  tQ.  Then  for 

all  t  -  t  ,  we  have 
o 


E{e(t)x' (t) }  =  -  Z*( t) 


(4.3.46) 


Proof:  At  t  =  tQ,  we  have  from  (4.3.44),  (4.3.33)  and  (4.3.31)  that 

E(e*(t  )x'(t  )}  =  {I  -  Z  C'(t  )(C.(t  )Z  C'(t  )_1C9(t  )}{x  x'-E  x(t  )x’(t  )} 
o  —  o  — n  — c r-2  o  ~  2  o  —o~l  o  —2  o  — o~ o  —  o  —  o 


■  -  i  «o> 


(4.3.47) 


Using  (4.3.45) 

dE<>  (t)x'  (t)  > 

_ 


and  (4.3.1)  we  have 

-  (A(t)-V2(t)C2(t)-L*(t)C1(t))E{e'!:(t)x,(t)} 

+(V*(t)C_(t)-I  )R(t)+E{e*(t)x'(t)}A’(t) 

~  2  - n  —  _  _  _ 

=  (A(t)-R(t)C^(t)A_1(c)C2(t))E{e*(t)_x,(t)} 

-Z*(t)C^(t)2_1(t)C1(t)E{e,'f(t)x’(t)} 

-Z*(t)C^(t)4_1(t)C2(t)E{e*(t)x,(t)}+E{e*(t)x,(t)}A,(t) 

+7*(t)C^(t)A"1(t)C2(t)R(t)-R(t)+R(t)C^(t)A"1(t)C2(t)R(t) 

(4.3.48) 
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Let  us  define 

D(t)  =  Z*(c)  +  E{e*(t)x'(t)}  .  (A. 3.49) 

By  (4.3.31),  (4.3.47)  and  (4.3.48)  we  have 

D(t)  =  (A(t)-R(t)C’(t)A"1(t)C2(t))D(t)-£*(t)(C|(t)S"1(t)C1(t)+C^(tU"1(t)C2(t))- 
D(t)+D' (t)A(t) 


D(t  )  =  0 
—  o  — 

(4.3.50) 

A 

£.  (t)  is  the  unique  minimal  sequence  of  $  and  s  is  well  defined.  (4.3.50) 

o 

implies 

D(t)  =  0  ;  tit  (4.3.51) 

and  (4.3.46)  follows. 

Theorem  4.3.4:  Let  e;  (t),  t  £  tQ,  be  described  by  (4.3.45),  and  v(t), 

t  ^  t  be  given  by  (4.3.1)  with  u(t)  =  0 ,  t  -  t  .  Then  for  all  tit: 
o  —  —  o  o 

E{eX(t)x{(s)}  =  0  ,  sc  [tQ,t)  ;  E{e*(c)^(s)}  =  0  ,  s  c  [tQ,t] 

(4.3.52) 

JProaf:  By  (4.3.45)  and  (4.3.1)  and  the  properties  of  Gaussian  white  noise: 
(t)y|(s)}  =  E{^  (t)x' (s)  }g|(s)+E{£*(t)n^'  (s) ) 

=  [Efe  (s)x' (s)  (s)+L*(s)g(s) ]  s  e  [tQ,t) 

(4.3.53) 

where 

A(t)  =  A(t)  -  V*(t)C2(t)  -  L*(t)C  (t) 


(4.3.54) 


-120- 

t,s)  is  the  fuadasesal  matrix  associated  with  A(t) .  Xoa  a  g 
lesna  4.3.3,  (4.3.53)  and  (4-3.27) 

Ele* (t)v*(s) )  -  £-(t,s) [-  S* (s)C' (s>  -5-  T* (s)C' (s)  1  =  0  .  (4.3.55) 

Similarly  fcr  s  e  ve  havejby  using  cocoatifeility} 

E*e*{t)v|(s)}  =  ^(t,s)Ele*(t)£*(s:Jc;(s)  =  (s)C* (s)  =0  .  (4.3.56) 

Tee  above  teecrem  implies  that  for  zero  control,  the  optimum  class  of 
observers  and  their  associated  estimators  v-ili  ail  generate  (a.s.)  the  con¬ 
ditional  mean  estimates  of  je(t) .  The  results  also  holds  if  u(t)  is  a 
nonzero  but  known  deterministic  control  function,  because  we  can  always 
subtract  its  deterministic  contribufciou.  The  case  where  the  control  is 
generated  via  a  special  class  of  feedback  law  will  be  considered  in 
chapter  V.  Sote  that  we  obtain  the  Kalman  Filter  as  a  special  case  when  we 
set  C,(c)  -  0  (4.3.29). 

4.4  Asvnptotic  Behavior  of  Estimators 

Let  us  first  consider  the  asymptotic  behavior  of  classes  of  observers 
and  associated  estimators  for  a  deterministic  system  j.  Then,  we  shall 
consider  the  asymptotic  behavior  of  optimum  classes  of  observers  and 

associated  estimator  for  the  stochastic  system  g^'. 

c 

Definition  4.4.1;  The  system  is  detectable  at  t  if  there  exists  an  ob¬ 
server  &^C(L),  T(t)  z  £C,  and  its  associated  estimator  &ic(L): 

IV  i 

1(0  =  (T(t)(A(t)-L(tjC(t})P(t)+T(t)P(t))z(t)+[T(t)(A(t)-L(t)C(t))V(t) 
lc 

&T  (L):  +T(t)V(t)+T(t)L(t)hl(t)+T(t)B(t)u(t) 

w(t)  =  P(t)z(t)+V(t)x(t)  ;  z(t)  z  S_  =  (T(i)a  |  ct  z  Rn} 


(4.4.1) 


such  char  for  all  z(r)  e  S_,  w(t)  -  s(t),  as  c  -  *.  The  system  8,  is  said 

T  X 

co  be  detectable  if  it  is  detectable  at  t  e  (-»,*). 

He  shall  say  a  stochastic  system  S”  to  be  detectable  if  its  determi- 

c 

aistic  analog,  S;>  is  detectable. 

c 

Theorem  4.4.2:  The  system  is  detect;  -1_  if  and  only  if  there  exists 
lc  c 

an  observer,  6„  (L),  3y,  which  is  uniformly  asymptotically  stable. 

Proof;  The  estimation  error  by  using  any  observer,  u^(L), 

T(t)  £  ev»  and  its  associated  estimator  is  given  by  (see  equation  (4.3.5)) 

e(t)  =P(t){z(t)  -  T(t)x(t)  4  ?(t)j?(c,T;zo)  (4.4.2) 

where  z(t,r ;z^)  satisfies  (see  equation  (4.2.6)  and  theorem  4.2.2) 

jKt,?;^)  =  [T(t)(A(t)-L(t)C(tj)?(t)+T(t)P(t)]z(t,T;2Q)  ; 

Kc.t;^)  -  z,  e  S.  .  (4.4.3) 

Let  us  first  assume  that  there  exists  sone  L(t)  e  M  and  V(t)  e  M 

—  nn  —  *  urn 

such  that  an  observer  &^C(i) ,  T(t)  E  is  uniformly  asymptotically 
stable;  then  for  all  t  and  z  e  S  ,  z(t,r;z  )  +  0  as  t  +  “.  From  (4.4.2) 

— O  T  —  — O  — 

we  conclude  that  8^  is  detectable.  Conversely,  if  .r,e  system  is  de¬ 
tectable,  then  there  exists  an  observer,  6^C(L),  T(t)  s  T~,  such*  that  the 
output  of  its  associated  estimator  will  give  exact  asymptotic  estimates 
independent  of  when  we  initiate  the  observer  state;  i.e.,  for  all  i,  and 

z  e  S 
— o  T 

e.(t)  =  PCt^t,!;^)  -*■  £  as  t  +  «  (4.4.4) 

where  is  given  by  (4.4.3).  We  may  assume  P(t)  to  be  of  full 

rankj  thus  (4.4.4)  implies  that  the  system  (4.4.3)  is  uniformly  asymptotically 


1 
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2.C  —  9*c 

stable,  and  so  the  observer  (L) ,  T(c)  e  o-,  is  uniformly  asymptotically 

X  V 

stable. 

A  linear  system 


x(t)  *  A (t)x(t) 


(4-4.5) 


is  said  to  be  exponentially  stable  if  there  exists  c^,  >  0  such  that 


il4<t,T)Ji  *  a3e 


c2 I C_T ! 


(4.4.6) 


where  c>  (t,x)  is  the  fundamental  matrix  of  A(t).  Ke  also  say  that  the 

'A 

matrix  A(t)  is  exponentially  stable.  Theorem  4.4.2  ielii.es  detectability 
to  the  structure  of  the  observers.  Since  exponentially  stable  implies 
uniform  asymptotic  stability  and  vise  versa  (48),  the  above  lemma  implies 

c 

that  the  system  is  detectable  if  and  only  if  there  exists  an  observer 
Xc  c 

©T  (L),  T(t)  e  such  that  the  error  of  estimates  (in  the  noise  free 
lc  c 

case)  by  using  &T  (L) ,  T(t)  e  ffy^has  the  bound 


1 1 e (t) j  j 


-o„  t-t 


iaje 


(4.4.7) 


lc 

where  tQ  is  the  initial  time.  We  may  call  such  an  estimator  6  (L)  an 

[491 

exponential  estimator. 

Theorem  4.4.3:  If  there  exists  V„(t)  z  M  of  rank  nu  and  L. (t)  z  M 
-  — /  nm^  Z  — l  nra^ 

such  that  (A(t)  -  ~  (t)C^(t) )  is  exponentially  stable,  then 

the  equivalent  classes  of  observers  &^C(L) ,  T(t)  z  3^,  where 
V(t)  =  [0  !  (t)l  and  L(t)  =  [L^(t)  •  arbitrary»  are  all 

uniformly  asymptotically  stable  and  so  g.“C(L),  T(t)  z  (Ty,  will  yield  ex¬ 


ponentially  consistent  estimates. 


-123- 


Proof:  Ler  us  consider  the  class  of  _V(c)  of  the  fort:  (4.3.4)  witn  rank 
V,(t)  =  The  error,  e(c)  k  v(t)  -  x(t),  of  estimates  by  using  V(t) 
within  this  class  is  given  by  (see  also  (4.3.22)) 


i(c)  =  (A(t)  -  V,(c)C,(t)  -  L1(c)C1(t)>e(t)  .  (4.4.8) 

Sy  assumption,  there  exists  V_(t'  s  M  of  rank  mu  and  L,  (c)  *  M  ,  such 

*  02,  x  —1  nm^ 

that  (A(t)  -  V9(t)C7(t)  -  L., (t)Ci(c))  is  exponentially  stable,  thus  the 
theorem  follows  from  (4.4.8)  and  theorem  4.4.2. 

Theorem  4.4.3  gives  us  a  sufficiency  test  for  detectability;  it  also 
indicates  how  we  can  construct  an  exponential  estimator. 

For  the  stochastic  system  S?,  the  class  of  minimal  order  optimum  ob¬ 
servers  and  their  associated  estimators  are  given  by  (4.3.39)  and  (4.3.42); 
the  optimum  error  covariance  2  (t)  is  given  by  (4.3.29).  We  shall  now 

investigate  the  asymptotic  behavior  of  the  class  of  minimal  order  optimum 
* 

estimators  via  E  (t). 

Theorem  4.4.4:  The  matrix  function  E*(t)  will  remain  bounded  for  all 

t  e  [t  ,°°]  if  and  only  if  there  exists  V.(t)  e  M  and  L  (t)  e  >i 

°  ~ 2  nm^  —1  nm^ 

such  that  (A(t)  -  ^(t)^^1-)  “  L^(t)£^(t))  is  exponentially  stable. 

Proof:  This  follows  immediately  from  theorem  4.3.2.  The  reader  is  re¬ 
ferred  to  the  proof  in  the  discrete  analog  for  the  detailed  argument.  Using 
theorem  4.4.2  and  4.4.3,  we  see  that©^C(L*),  T(t)  e  3^*,  is  uniformly 
asymptotically  stable. 

Corollary  4.4,5:  If  (A(t),jl(t))  is  uniformly  completely  observable,  i.e., 
there  exists  »  >  t  >  0  such  that 
ft+T 


M(t)  = 


t 


i^(o>t+T)C' (o)C(o)^A(o,t+T)do  ;  t  e  [t^0®]  (4.4.9) 


( 
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has  rank  n,  then  mere  exists  L(t)  £  M  such  char  (A(t)  -  L(c)C(t))  is 

—  IU3  —  —  — 

exponentially  stable. 

Proof:  Apply  theorem  4.4.4  to  the  special  case  when  =  0;  i.e.,  all 

observation  channels  are  corrupted  by  w.iite  Gaussian  noise.  The  optimum 

error  covariance  will  remain  bounded  if  and  only  if  there  exists  L^t)  t  J5 

—  nn 

such  that  (A(r)  -  L(t)C(t))  is  exponentially  stable.  If  (A(t),C(t))  is 

uniformly  completely  observable,  the  optimum  error  covariance  wili  remain 

T50] 

bounded  for  all  t  e  {tQ,®],  and  so  the  corollary  follows. 

Let  us  consider  the  time  invariant  case  where  A,  C^,  and 
are  constant  and  bounded  matrices. 

Lemma  4.4.6:  If  the  pair  (A,C)  is  observable,  then  the  pair 


is  also  observable. 

Proof:  Construct  the  matrix 

m(c  ,t)  =  f  6!(<j,t)[c;  :  Cl] 

-  o  ;t  “A  -1  .  -Z 

o 


— A (°  *  ~ ) d° 


±'A (o  ,  '  )C^C1£^(0  ,  T )  do  + 
o 


do 


d£2iA(o,T) 

do 


do 


(4.4.10) 


Let  jx  t  R  such  that  x.'M(t  »t)x  =  0 * 
c  e  [ tQ ,  t  ] : 


then  from  (4.4.10)  we  have  for 


ci1a(°>t)x  =  0  *  C  ^  (o,i)x  =  2.  e  Rm  (4.4.11) 

— i  A  —  ~Z  A  — 

where  is  a  constant  vector.  Suppose  that  x.  i  Of  let  x  =  £  (t  ,t)jc?  then 

x  f  0;  let  x,  =  c.(t,T)x,  t  >  t,  >  t  ,  also  x,  i  0.  Since  A  is  constant, 
— o  —  — 1  —A  —  1  o  —1  —  — 
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ve  have  from  (4.4.11)  that 


=  =  £  ;  c  €  K'-] 


(4.4.12) 


=  ^(o.s^^.t)*  =  C^ic  4-  tx  -  tc>t1)cA(t1>t)x 
*  %=:A(a  +  tx  -  tQ,T)x  =  v  ;  c  s  [to>:  -  ^  +  tQ] 

(4.4.13) 

Thus  xo»  are  indistinguishable  by  observing  the  output  in  the  nonzero 
interval  [t^.r  -  t^  +  tQ].  This  contradicts  the  assumption  that  (A,C)  is 
observable. 

* 

Lemma  4.4.7;  Let  =  ();  the  solution  of  (4.3.29),  denoted  by  Z^  (t;j))  will 

* 

reach  a  steady  state  Z  which  satisfies 

1  it  i  lit 

0  =  (A-R  C^£\A)Z  +  Z  (A-R  CJA"  CjA)  ’  -  Z  (A’C^  C-^E 


..*-1 


+  R-R  C’A  C2R  ;  A  -  C2R  C2  >  0 


(4.4.14) 


if  and  only  if  there  exists  V2(t),  L.  (t)  such  that  (A  -  V2(t)C?  -  L.^ (t)^) 
is  exponentially  stable. 

A 

Proof:  Let  us  consider  £  (t;0)  as  a  minimal  function  with  respect  to  the 

* 

solution  set  .  With  the  assumption  that  Z  =  (),  we  have  _Z  (t  ;0)  =  () 
o  ° 

from  (4.3.25),  and  so 


I  (t;0)  = 


i(t,T){(ITi-V2(T)C2)R(ITi-V2(T)C2),+L1(x)£  L1'(T))o,(t,T)dT 


(4.4.15) 

.* 


where  jKt.i)  is  the  fundamental  matrix  associated  with  (A  -  V.2(t)C9  -  L^Ct)^), 

•k  _  ■> 

2(t),  L1( 


it  -s.  it 

and  V  (t),  L.  (t)  are  given  by 
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vj(c)  -  (E* (t;0)c^  -*  R  cpi'1  ;  C  -  Eo 
L*(c)  =  2*(t;0)C^  5.’1  :  t  >  to  . 


(4.4.16) 

(4.4.17) 


Let  \g(t)  =  V*(t  +  o),  L°(t)  =  L*(t  +  a),  and  £°(t,r)  be  the  fundamental 
matrix  associated  with  (A  -  VgCt)^  -  Clearly  we  have 


6C(t,T)  =  6(t  +  o,t  +  o) 


(4.4.18) 


Let  l°(t;0),  t  -  t  be  the  solution  of  (4.3.25)  with  £  (tQ;0)  thus 

T®(t*0)  c  B  .  Since  2*(t)  is  the  minimal  function  with  respect  to  Bt  ,  we 
“  “  Co  ° 

have 


(t;0)  1  2° (t;0) 


t  2  t 


(4.4.19) 


Also  we  have  from  (4.4.18)  and  the  definition  of  that. 

:r(t-o;0)  =  f  V(t-o,T){(In-V°(T)C2)R(In-V°(t)C2)+L°(T)S  L°*(T)}6°J(t-o,T)dT 


"  i  o(t,Y){(In-V2(Y)C2)R(In-V2(Y)C2)+L1(Y)ii  L1(Y)b,(t,Y)dY 


* 

=  2  (t) 


(4.4.20) 


Combining  (4.4.19)  and  (4.4.20)  we  have 

2*(t;0)  >  2° (t  -  o; 0)  >  2*(t  -  o; 0)  .  (4.4.21) 

The  lemma  follows  from  theorem  4.4.4  and  the  monotone  nondecreasing  nature 
of  Z  (t;0)  as  t  increases  (4.4.21). 

Theorem  4.4.8:  For  all  2^  >  0,  the  solution  of  (4.3.29),  denoted  by 
Z*(t;Z  )  will  reach  a  steady  state  2*  which  satisfies  (4.4.14)  if  and  only 
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there  exists  ^(t),  sucn  chat  (A  -  ”  — l^c^— 1^  *s  exPoneRC^a-^y 

stable. 

Proof:  From  (4.3.29)  and  (4.3.25)  we  have 


1  <v4>  -  i  <V$>  -  4  -  WW  £A, J  s 


(4.4.22) 


There  fore,  from  (2.6.21)  we  deduce  that 


S*(t;I  )  1  E*(t;0)  ,  t  f  [t  ,»]  .  (4.4.23) 

—  O  —  —  O 

* 

Using  the  minimal  property  of  £  (t;£  ),  we  have 

Oil*  (t;^)  -  E*(t;0)  *  6(t,to)Z^to;ni,  (t,tQ)  (4.4.24) 

—  -• 

where  d(t,t  )  is  the  fundamental  matrix  associated  with  (A  -  V  (t)C„  -  L1(t)C1) 
”  o  -  l  — 1  —l 

«-  ^ 

and  V^t),  L^(t)  are  given  by  (4.4.16)  and  (4.4.17).  6^(t,tQ)  is  exponen¬ 

tially  stable  if  and  only  if  there  exists  V^t),  L^(t)  such  that 
(A  -  V^(t)£2  "  is  exponentially  stable.  Using  lemma  4.4.7,  we 

obtain  the  theorem  easily. 

From  lemma  4.4.6,  and  corollary  4.4.5,  we  see  that  observability  of 

5V  st 

the  pair  (A,C)  is  sufficient  to  assure  that  E.  (t;E^)  L  satisfying 
(4.4.14)  where  E  >  0  is  arbitrary. 

— 0  — 

4.5  General  Discussion 

In  this  chapter,  we  considered  the  estimation  of  deterministic  and 

stochastic  systems  using  the  observer  approach. 

In  the  deterministic  case,  sufficient  conditions  for  existence  of 

exponential  estimator  have  been  derived;  such  estimators  can  be  realized 

by  an  observer  &^C(L) ,  T_(t)  c  <1^,  which  is  asymptotically  stable,  and  its 

lc 

associated  estimator  g,  (L) . 
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In  the  stochastic  case,  the  minimal  order  optimum  observer  and  its 

estimator  are  described  in  detail  in  Figure  4.4.  The  optimum  error  co- 
* 

variance,  (t),  is  given  by  (4.3.29).  Asymptotic  behavior  of  the  minimal 

order  optimum  observer  is  investigated  via  the  optimum  error  covariance 
*  : k 

E.  (t).  Necessary  and  sufficient  condition  for  Z  (t)  to  be  uniformly 
bounded  have  been  established.  The  condition  is  related  closely  to  the 

c 

structural  property  of  the  system  under  consideration. 

In  the  following,  we  shall  discuss  different  points  which  are  rele¬ 
vant  to  the  whole  development  in  this  chapter. 

(A)  Unbiased  Estimates  and  Observer-Estimator  Structure 
c 

Let  §2  be  a  stochastic  system  described  by  (4.3.1)  with  ]j(t)  =  0. 

Let  an  unbiased  estimator  £  be  given  by 


w(t)  =  |  H(t,i)x(T)d-  +  V(t)^(t) 


(4.5.1) 


where  H(*,-)  is  an  nxm  matrix  whose  elements  are  differentiable  in  both 
arguments.  Since  E|w(t)|  =  E|x(t)|  ,  from  (4/3.1)  and  (4.5.1)  we  have 


!  H(t,x)C(T)b  'x,t  )x  dt  +  V(c)C(t)o,(t,t  )x  =  *.(t,t  )x  (4.5.2) 

—  — O  — O  — A  rt  — ^  rt  — o 


where  t^(t,i)  is  the  fundamental  matrix  associated  with  A(t) .  The  structure 

of  the  estimator  should  be  independent  of  the  mean  of  x(t  ),  x  ,  thus 

—  o  — o 

(4.5.2)  implies 


H(t,T)C(x)l  (r.t  )d-  +  V(t)C(t)i  (t,t  )  =  A  (t,t  )  •  "'.5.3) 


Differentiate  both  sides  of  (4.5.3)  in  respect  tot. 

H(t,t)C(t)+[  i^t,T)  c(T)iA(T,t)dT+V(t)C(t)+V(t)C(t)+V(t)C(t)A(t)  =  A(t)  . 


o 


(4.5.4) 
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Multiplying  both  sides  of  (4.S.4)  by  w(t)  and  taking  expectations 

I  {(H(t,t)C(t)+V(t)C(t)+V(t)C(t)+V(t)C(t).A(t)-AJCt))H(t,T)+  °H-^  ^}E{jr(i)  )dx 
'  t 

o 

+(H(t,t)C(t)+V(t)C(t)+V(t)C<t)+£(t)C(t)-A(t))V(t)E{2(T)}=Q  (4.5.5) 

(4.5.5)  is  satisfied  if  H(t,T)  and  V(t)  satisfy 

f  (G(t)H(t,T)  +  5H^>T^(T)dT  =  -  G(t)V(t)£(t)  (4.5.6) 

Co 

where  ^(t)  is  a  m-vector  valued  function  of  t;  and 

G(t)  =H(t,t)C(t)  +  V(t)C(t)  +  V(t)C(t)  +  V(t)C(t)A(t)  -  A(t)  ,  (4.5.7) 

Let  us  denote  w  (t)  =  [  H(t,T)_y_(t)dT;  we  have 

-i  t 

o 

wx(t)  =  H(t,t)Z(t)  +  J* 

Co 

=  H(t,t)x(t)  -  G(t)w1(t)  -  G(t)V(t)x(t)  .  (4.5.8) 

The  unbiased  estimator  is  realized  by 

w(t)  =  -  G(t)wx(t)  +  (H(c , t)  -  G(t)V(t))^(t) 

£:  (4.5.9) 

w(t)  =  w^t)  +  V(t)Z(t) 

By  some  transformation  of  coordinates,  the  unbiased  estimator  £  can  be 
realized  by 

A(t)  =  F(t).z(t)  i-  D(t)^(t) 

£’  :  (4.5.10) 

w(t)  "  P(t)^(t)  +  V(t)_y(t) 
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Siuce  f '  an  unbiased  estimator,  we  have  at  t  =t 


*  l(t)z( t  )  +  V(t  )C(c  )x 

o  —  o  —  o  —  c  —  C  — 0 


(4.5.11) 


If  T(t  )  £  ~(C(t  >;u,s,n)  and  ?{t  )  is  such  that 

—  — *  r’%  —  n 


£«„«**«>  +  2<t0)c«cc)  -  4  . 


(4.5.12) 


iacnoy  setting  ~(t  )  =  T(t  )x  ,  (4.5.11)  is  cieaxl?  satisfied.  Also  ve 
~  o  —  o  —o 

cave  E;v(t)  -  =  E-"x(t)  •  thus 

•c 

?(c)  )s4  r_(t,r)D(-.)C(T)5.(-,t  )>:  dtd-V(t)C(t)?.  (t,t  )sc 

—  »  q  o  o  — *  —  —  — o  — o  —  —  — A  c  — o 


=  t,U,t  }x 
a  o 


(4.5.13) 


ace  structure  of  the  estimator  is  reauired  to  be  independent  of  ,  there- 

— o 

fore  (*.5.13)  iqdies 


?(t)T(t)  d-  V(t)C(t)  =  i 


(4.5.14) 


vhere  T(:)  is  given  by 


T/t)  -  i-(t,t  )7<t  )i.(t  ,t)  +  :_(r,T)D(OC(')*.A(T,t)dT  .  (4.5.15) 

^  o  o  — A  O  _r  -  -  —A 

*  t 

o 

3y  comparing  vich  (4.2.4)  and  theorem  4.2.2,  such  an  estimator  can  be 

realized  by  an  observer  0*C(L),  T(t)  £  Uy  and  its  associated  estimator 
ic 

S-^d),  vhere  L(t)  -  M  is  arbitrary. 
i  —  nm 

Thus  ve  see  that  the  concept  of  an  observer  is  in  some  sense  equiva¬ 
lent  to  the  concept  of  unbiased  estimator.  When  the  a  priori  distribution 
of  3c(tQ)  is  knovn,  the  minimal  order  optimum  observer-filter  gives  un¬ 


biased  minimum  uean  square  estimates;  whereas  if  the  a  priori  distribution 


of  _x (c0)  is  unknown,  the  minimal  order  optimum  observer-filter  will  be 
an  asymptotically  unbiased  minimum  mean  square  estimator. 

(B)  Estimation  for  linear  System 
The  observer  theorem  introduced  in  this  chapter  generalizes  an 4 

unifies  estimation  theory  for  deterministic  and  stochastic  systems.  For 
both  deterministic  and  stochastic  cases,  the  structure  of  the  estimators 
are  the  same.  In  the  deterministic  case,  we  are  to  find  certain  parameters, 
^(t),  L^(t),  so  as  to  obtain  exponentially  consistent  estimates,  whereas  in 
the  stochastic  case,  the  optimum  choice  of  V^O  and  L^(t)  is  specified  by 
the  noise  statistical  law  and  tho  detailed  structure  of  the  system.  Thus 
we  see  that  in  the  deterministic  case,  qualitative  theory  should  be  used 
in  designing  well-behaved  observer-estimator;^9^  whereas  in  the  stochastic 
case,  optimization  technique  can  be  applied  to  derive  the  class  of  minimum 
order  optimum  observer-estimator. 

(C)  Kalman  Filtering  Technique 

We  can  also  s  -Ive  the  stochastic  problem  in  section  4.3  by  using  the 

Kalman  Filtering  Approach.  Let  us  consider  the  system  with  u(t)  =  £. 

Let  T(t)  e  M  .  .  such  that 

n(n-m^) 

I(t) 

•  *  • 

c2(t) 

is  of  full  rank.  Define 


xx(t)  =  T(t)x(t) 


(4.5.16) 


‘This  approach  was  suggested  by  I.  B.  Rhodes. 
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Then  we  have 


x(t)  =P(Ox1(t)  +V2(t)x2(t) 

where 


[P(t)  :  v2(t>] 


T(t) 


We  have  the  equation  for  :c^(t)  and  v^(t) 


xx(t)  =  (T(t)+T(t)A(t))x(t)+T(t)i(t) 


(4.5.17) 


(4.5.18) 


=  (T(t)P(t)+T(t)A(t)P(t))x1(t)+(T(t)V2(t)+T(t)A(t)V2(t))- 

l1(t)-C1(t)V2(t)v2(t)  =  Ci(t)P(t)x1(t)+n(t)  (4.5.19) 

Since  £0(t)  can  be  observed  exactly,  we  can  assume  it  is  known.  Now 
apply  a  Kalman  filter  to  the  system  (4.5.19):  the  best  mean  square  esti¬ 
mate  of  x-^(t)  is  given  by 

ix(t)  -  (T(t)P(t)+T(t)A(t)P(c)-Z(t)P,(t)C1(t)S_1(t)C1(t)P(t))x1(t) 

+  (T(t)V  (t)+T(t)A(t)V  (t))^.-(t) 

Z  (4.5.20) 

+Z(t)P,(t)C1(t)^"i(t)(il(t)-C1(t)V2(t)l2(t)) 

x  (t  )  =  T(t  )x 
— i  o  —  o  — o 


and  I(t)  satisfies 


l(t)  =  (T(t)_P(t)+T(t)A(t)P^(t)^(t)+f^(t)  (T(t)P^(t)+T(t)A(t)P^(t) ) ' 

+T(t)R(t)T' (t)-2 (t)P’ (t)C{(t)2"1(t)C1(t)P(c)|(t) 

:(t  )  =  T(t  )ZT' (t  ) 
o  —  o  —o—  o 


(4.5.21) 


The  estimate  of  x(t)  is  given  by 


x(t)  =  P(t)x1(t)  +  V2(t)i2(t) 

The  estimation  error  covariance  matrix  is  given  by 

Z (t)  =  P(t.)|(t)P'  (t) 

Therefore 

|(t)  =  P(t)i(t)P,(t)+P(t)f(t)P'(t)+P(t)|(t)P,(t) 

=  (P  (t)+P^(t)T(t)P^(t)+P^(t)^(t)A(t)F^(t))^(t)P^’  (t) 

+  P(t)f(t)(P(t)+P(t)+T(t)P(t)+P(t)T(t)A(t)P(c))' 

+  P(c)T(t)R(t)T,(t)P,(t)-P(c)|(t)P,(t)C’(t)^“1(t)C1(t)P(£)Z(t)P,(t) 

-  (A(t)-V2(t)C^2(t))E^(t)+x(t)  (A(t)-_V2(t)^2(t)) ' 

+  (In-V2(t)C2(t))R(t)(In-V2(t)C2(r)),-Z(t)C’(t)fi"1(t)C1(t)Z(t) 

(4.5.24) 

The  initial  condition  is 

Lit)  -  a  -  V_(t  )C,(t  ))E  (I  -  V_(t  )C,(t  ))'  .  (4.5.25) 

o  ~n  2  o  2  o  ~o  — n  2  o  2  o 

We  note  that  the  error  covariance  depends  on  V2(t)  which  must  satisfy 

C_(t)V.(t)  =  I  .  (4.5.26) 

—2  —2  — m0 

To  find  the  minimum  mean  square  estimates,  we  have  the  optimization  problem 
of  choosing  V2(t)  satisfying  (4.5.26)  and  yielding  the  "least"  nonnegative 
definite  E_(t).  Note  that  (4.5.24),  (4.5.25)  is  the  same  as  (4.3.25)  with 

Lx(t)  =  Z(t)C^(t)£_1(t) 


(4.5.22) 


(4.5.23) 


(4.5.27) 


1 
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One  can  easily  show  that  Che  optimum  estimator  derived  by  using  the 
Kalman  filtering  approach  is  a  minimal  order  optimum  observer-estimator. 
Before  comparing  the  merits  of  Kalman  filtering  approach  and  observer- 
estimator  approach  as  developed  in  this  chapter,  the  author  would  like  to 
point  out  the  falacy  of  an  initiative  conception  by  using  the  Kalman  fil¬ 
tering  approach.  This  is  best  explained  but  giving  a  specific  example. 
Consider  a  linear  time  invariant  system  described  by 


x.(c)  0  llfx  (t)l  R.  (t)' 

+ 

x2(t)  -a^  -a2  x 2 (t)  52(t) 

The  observations  are 


(4.5.28) 


y1(t)  =  (0  ljx(t)  +  n(t)  (4.5.29) 

y,(t)  =  [1  0]x(t)  .  (4.5.30) 


The  noise  statistical  laws  are  assumed  to  be  known: 


F.{i  i(T)d-}-0  ;  E 


{((  i<T)d-)(f  I(o)do)  }  =  1  (4.5.31) 

1  Jo  'o  0  t  t 


E{ j  n(T)d-}  =  0  ;  e{(|  n(x)d"]  |  =  qt 


(4.5.32) 


Assume  that  the  estimation  process  has  started  at  -»,  and  our  objective  _ 
now  is  to  find  the  conditional  mean  estimate  of  the  state.  One  "intuitive" 


argument  using  the  Kalman  filtering  approach  will  be  as  follows.  From 
(4.5.30),  we  see  that  we  have  exact  observation  in  x^(t),  therefore  we  can 
assume  x^(t)  is  known.  From  (4.5.28)  and  (4.5.29),  we  have 
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x2(t)  =  -  a^x, (t)  -  a2x2(r)  +  C,(c)  (4.5.33) 

y^t)  =  x2(t)  +  n(fc)  .  (4.5.34) 

Since  the  system  is  linear  and  the  noises  are  Gaussian,  thus  to  find  the 
unbiased  mean  square  estimate  of  x2(t),  we  may  apply  Kalman  filter  to 
(4.5.33)  and  (4.5.34).  The  error  variance,  e,  in  the  steady  state  will 
satisfy  the  algebraic  equation 

2 

e  +  2a^qe  -  r2q  =  0  .  (4.5.35) 

Therefore  the  error  variance  is  equal  to 

9  2 

e  =  /a“q  +  r2q  -  >0  .  (4.5.36) 

One  may  make  the  conclusion  that  the  Kalman  filter  for  (4.5.33)  and  (4.5.34) 
will  give  us  the  unbiased  minimum  least  square  estimates,  and  the  minimum 
mean  square  error  is  given  by  (4.5.36).  Unfortunately,  this  conclusion  is 
in  general  false:  the  reason  for  this  is  that  the  Kalman  filter  for  (4.5.33) 
and  (4.5.34)  give  us  the  estimate 

x2(t)  =  E{x2 (t) | F (y j (t ) ; r  e  [tQ,t))}  (4.5.37) 

whereas  the  estimate  we  are  looking  for  is 

x2(t)  =  E{x2(t) | F (y2 (t) ;t  e  [tQ,t] ,y1 (s) ;s  e  [ , t ) ) }  (4.5.38) 

and  in  general  we  have  the  inclusion  of  o-algebra 

F(y1(x);t  t  [to,t))C  F(y2(T);x  e  [to,t]  .y^s)  ;s  e  [tQ,t))  (4.5.39) 

To  proceed  with  the  example,  let  us  define 
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xlk(c)  =  [-k  l]x(t) 


(4.5.40) 


where  k  is  an  arbitrary  number.  Using  (4.5.30)  and  (4.5.40),  we  have 


x(t) 


J° 


'O' 

J ** 


(t)  + 


(4.5.41) 


Taking  the  derivative  of  (4.  5.  40)  and  using  (4.  5.  28),  we  have 
xlk(t)  =  -(k  +  a2)x2(t)  -  a^U)  +  ?2(t)  -  kC^t) 

=  -(k  +  a2)x1k(t)  -  (k(k  +  &0)  +  a1)y2(t)  +  52(t)  “  kCj^t)  .  (4.5.42) 

The  observation  (4.5.29)  becomes 

yl(t)  =  xlk(t)  +  ky2(t)  +  n(t)  .  (4.5.43) 

Define 


yll'(t)  "  yl^C^  ~  ky2(t)  =  xlk^^  +  ‘  (4.5.44) 

Since  y2(s),  s  c  [ tQ , t ] ,  is  known  at  t,  by  applying  the  Kalman  filter  to 
(4.5.42)  and  (4.5.44),  we  have  the  steady  state  error  variance,  e^,  for 
the  unbiased  least  square  estimate  of  x^(t)  satisfying  the  algebraic 
equation 

e^  +  2(k  +  a2)qek  “  ^^2ri  +  r2)  =  0  (4.5.45) 

and  so 

=  /(k  +  a9)2q2  +  q(k2r1  +  r2)  -  (k  +  a2)q  >  0  .  (4.5.46) 

To  find  the  corresponding  estimate  in  ic,  we  have 
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x(t) 


roi 


(t>  + 


(4.5.47) 


where  x  (t)  is  the  estimate  given  by  the  Kalman  filter  for  (4.5.42)  ar.d 

■Uv 

(4.5.44).  The  corresponding  error  variance  for  x2(t)  is 


E{(x2(t)-x2(t))2}  =  E{(xlk(t)-xlk(t)+ky2(t)-ky2(t))2}  =  E{ (xlk(t)-xlt(t))2> 


lk' 


=  e. 


(4.5.48) 


Clearly  e,  i  e  for  almost  all  k  ranging  from  -»  to  =°.  One  may  then  attempt 

K. 

to  find  the  optimum  k  which  give  us  the  smallest  ek»  This  has  easily  been 
carried  out  by  using  differential  calculus.  The  optimum  value  for  k°  was 
found  to  be: 


k°  = 


2  2 
a2q 


a2q 


(q  +  r1)2  +  r](q  +  rl)  _q  +  rl 
Substituting  (4,5.49)  into  (4.5.46),  we  have  the  corresponding  error: 


(4.5.49) 


O  A 
e  e 


o  q 


2  <<!  +  t1)r2q 

«az!  +  — f: - qa2 


(4.5.50) 


and  clearly  we  have  the  strict  inequality  (r^  >  0,  q  >  0) 


V 


e  <  y  (q£2)  + 


(q  +  r.) 


i y  2 

—  r2q  -  qa2  <  /(qa2)  +  r2q  -  qa2  =  e  .  (4.5.51) 


The  inequality  (4.5.51)  indicates  that  by  applying  Kalman  filter  to  (4.5.33) 
and  (4.5.34),  we  do  not  obtain  the  best  mean  square  unbiased  estimate.  We 
note  that  the  optimum  value  of  k  depends  on  r^,  r2>  and  q  (assume  a2  is 
fixed  a  priori).  We  may  not  conclude  that  the  error  e°  is  the  minimum 
error  variance  because  we  only  consider  a  restricted  class  of  transformation 
in  x  (4.5.40).  The  only  way  to  check  whether  e°  is  the  minimum  error  variance 
is  to  appeal  to  the  projection  equation,  or  equivalently ,  the  Weiner-Hopf  equation. 
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Thersfore,  we  see  that  conceptually,  the  Kalman  filtering  approach 
is  by  no  means  simpler  chan  the  observer  approach  developed  in  this  chapter; 
because  one  may  find  it  hard  to  visualize  physically  why  one  transformation 
of  the  state  is  better  than  the  other  before  the  application  of  a  Kalman 
filter,  besides,  one  may  reach  false  conclusions  if  one  is  not  careful 
(see  example).  Note  that  one  approach  is  as  easy  as  the  other:  both  in¬ 
volve  a  deterministic  optimization  problem,  and  both  need  to  verify  that 
the  derived  solution  satisfies  the  projection  equation  before  we  can  con¬ 
clude  the  truly  optimum  nature  of  the  obtained  estimate.  In  terms  of 
derivation,  the  Kalman  filtering  approach  is  comparatively  simpler;  but 
personally,  the  author  thinks  that  the  class  of  asymptotic  unbiased  esti¬ 
mator  is  a  more  basic  conceptual  framework  to  many  estimation  problems. 

The  observer  approach  is  based  precisely  on  this  conception.  One  dis¬ 
tinguishing  advantage  of  using  the  observer  theory  approach  is  that  it 
reveals  the  detail  structural  properties  of  the  optimum  estimator.  This 
allows  us  to  investigate  in  detail  the  asymptotic  behavior  cf  the  optimum 
estimator  in  terms  of  some  intrinsic  functional  behavior  of  the  system 
(section  4.4). 

(D)  Detectability  and  Observability 

We  note  that  observability  is  a  stronger  condition  than  detectability. 
In  section  4.4,  we  have  shown  that  detectability  is  a  necessary  condition 

k 

for  the  minimum  error  covariance,  Z  (t),  to  be  uniformly  bounded  for  al] 

t  -  t  .  In  the  time  invariant  case,  observability  is  sufficient  condition 
o 

_  k 

for  Z_  (t)  to  be  uniformly  bounded  and  for  the  existence  of  a  steady  state 

k 

value  of  Z  (t)  as  t  -<•  «•. 


and  the  augmented  matrices 


-140- 


Aa(t)  = 


_A(t) 

0 


„a 

v-* 


(O  =  [c( c)  :  i  ] 

.  m 


Ba(t) 


B(t) 

0 


(4.5.56) 


We  have  the  equations  for  the  augmented  system 


xa(t)  =  Aa(t)xa(t)  +  Ba(t)u(t)  +ia(t) 

&S  (4.5.57) 

£(t)  =  Ca(t)xa(t) 

We  can  apply  the  derived  results  to  the  system  §C.  Note  that  the  minimal 

Q. 

order  optimum  observer-estimator  has  dimension  n.  In  the  special  case 
when  G  is  a  constant  matrix,  we  can  easily  verify  that  the  results  obtained 
agree  with  those  obtained  by  Bucy  [52].  In  the  general  case,  the  results 
agree  with  Bryson  and  Mehra  who  considered  the  problem  using  the  weighted 
least  square  approach.  Application  of  the  derived  results  to  this  special 
class  of  problems  will  be  considered  in  detail  in  the  future. 

4.6  Perspective 

Qualitative  estimation  theory  for  the  deterministic  system  §  was  considered 
by  Luenberger  [35],  Johnson  [49].  Optimum  filtering  theory  for  stochastic 
linear  systems  was  firstconsidered  by  Wiener  [51].  Kalman  and  Bucy  [50] 
consider  the  special  case  of  estimating  the  state  of  a  Gaussian  Markov 
process  in  the  presence  of  nonaegenerate  Gaussian  white  observation  noise. 
Estimation  in  the  singular  situation  (i.e.,  noise  free  observation)  was 
considered  by  Root  [54].  (See  also  Van  Trees  [47]  for  detailed  bibliography.) 
Estimation  in  the  presence  of  colored  noise  only  was  considered  by  Bucy 
[52],  Mehra  and  Bryson  [53],  Geesey  and  Kailath  [55].  The  consideration 
in  this  chapter  provides  a  unifying  approach  to  linear  estimation  problems 
in  general.  This  approach  is  valuable  in  the  way  that  it  reveals  the  in¬ 
trinsic  structural  properties  of  the  estimating  device. 


The  asymptotic  behavior  of  the  estimator  was  not  investigated  in 
detail  in  the  literature  save  for  the  case  of  estimation  in  the  presence 
of  nondegene rati ve  Gaussian  white  observation  noise  [50].  In  this  special 
case,  the  asymptotic  behavior  of  the  optimum  estimator  was  investigated 
through  its  dual  relation  with  an  optimal  regulation  problem  [50].  The 
investigation  in  section  4.4  is  original.  In  this  contribution,  we  can 
study,  in  all  general  situations,  the  asymptotic  behavior  of  optimum 
estimators;  it  also  provides  the  concepts  required  for  qualitative  estimation 
theory  for  deterministic  linear  systems. 


C8T?5s5L  CG5THDL  Or  STOCHASTIC  LINEAR  SYSTEMS  KITS  SSKKS  DYNAMICS 


5il  latnxsatiea 

In  this  ctaprer,  w  are  mainly  concerned  with  the  problem  of  con¬ 
trolling  a  linear  svster  with  know a  dynamics,  under  else  assurpticn  that 
perfect  information  is  not  available.  To  have  the  problem  be  'completely 
general,  re  assure  that  there  are  inixjvra  iriv'p.g  disturbances,  and  partial 
observation  of  the  state  in  the  presence  (or  absence)  of  observation 
noise-  A  special  case  of  the  problem  vas  investigated  by  Joseph  and  Tou 
(Sri,  Gunckel  and  Franklin  [5Sj .  Kanban  [22  ] ,  [27],  mere-  they  assumed 
that  the  observation  noise  is  a  nontfegene rate  while  Russian  process.  In 
cv-r  investigation,  ve  assume  that  the  observation  noise  is  in  general  de¬ 
generate  vitbout  loss  ci  generality;  ve  shall  model  the  problem  as  one 
vhere  sene  output  variables  can  he  observed  ; - rfeztly  (noise-free)  vhile 


tne  otners  are  observed  in  the  presence  of  vhice  Saussian  noise-  This 
general  information  includes  [56],  158],  [27]  as  special  cases. 

The  structure  of  this  chapter  is  as  follovs.  In  section  5.2,  ve 
consider  the  estimation  problem  for  the  discrete  case  vhere  ve  are  2 llove a 
to  use  feedback  control.  Using  the  results  in  chapter  3,  ve  derive  a 
stochastic  difference  equation  for  the  conditional  mean  estimates  of  the 
current  state.  In  section  5.3,  ve  shall  state  the  stochastic  control 
problem  and  the  optimality  criteria  is  used  to  verify  the  optimal  solu¬ 
tion.  The  general  results  are  then  applied  to  a  special  case  vhere  the 


observation  noise  is  sequentially  correlated.  In  section  5.4  and  section 
5.5  ve  treat  the  continuous  analog  of  section  5.2  and  5.3.  The  results 
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can  be  summarized  as  the  Separation  Theorera.  In  section  5.6,  discussions 
of  results  and  indication  of  some  further  research  is  given. 

5.2  Estimation  with  Feedback  for  Discrete  Linear  Systems 
Consider  a  discrete  linear  system  described  by 


x(k  +  1)  =  A(k)x(k)  +  B(k)u(k)  +  £(k) 


v(k)  =  C(k)x(k)  +  n(k) 


(5.2.1) 


where  x(k)  e  Rn,  u(k>  t  Rr,  x(k)  £  Rm.  x(0),  £(k),  ri(k),  k  =  0,  1,  ... 
are  independent  Gaussian  random  vectors  with  statistical  law  giver  by 
(3.3.2)  tc  (3.3.4).  The  control  u(k)  is  feedback  in  nature.  Let  us  denote 
the  control  seauence  fev 


U(i,j)  k  iu(i),u(i  +  1)  , . . .  ,ti(j) }  i  >  j 


(5.2.2) 


The  observation  statistic  at  time  k  is  v*  ,  _ . (k) ,  where  the  subscript 

-ii(o,tv-Il 

U(o,k-l)  is  to  indicate  that  the  past  control  sequence,  U(0,k  -  1),  has 
been  applied  to  the  system.  The  accumulative  observation  statistic  at 
time  k  is 


YU(o,k-l)  00  {*(0)  ,:Zu(o)  (1)  ’  ‘  *  *  ’^U(o,k-l)  (k) 


(5.2.3) 


We  assume  that  the  control  is  of  the  form 


u(k)  *  i(k,Yu(o  )(k))  k  -  0,  1, 


(5.2.4) 


where  (k ,  * )  is  a  measurable  function  from  F(Y„,  .  ,.)  to  R  .  (5.2.4) 

—•  u(o,k-l) 

implies  that  the  control  is  a  function  of  past  accumulative  observation 
information.  In  the  following,  we  shall  denote  F(Yy^  j.  ^(k))  by 
F(k,U(0,k  -  1)). 
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Thc  information  revealed  by  the  accumulative  information  at  time  k 
about  the  dynamical  state  of  the  system  is  contained  in  the  sub-o-algebra 
F(k,U(Q,k  -  1)).  For  some  control  purposes,  the  detailed  knowledge  of 
F(k,U(0,k  -  1))  is  sufficient  but  not  necessary.  In  most  cases,  since 
the  knowledge  about  the  present  state  is  necessary  and  sufficient  for  de¬ 
signing  a  feedback  control  strategy,  then  the  knowledge  of  the  conditional 
distribution  of  the  present  state,  x(k),  is  necessary  and  sufficient  (see 
chapter  2,  section  2.2).  In  the  following,  we  shall  prove  that  the  condi¬ 
tional  distribution  of  x(k)  can  be  parameterized  by  some  finite  dimensional 
quantities. 

Theorem  5.2.1;  For  the  system  where  ut(k)  is  of  the  form  (5.2.4),  the 
conditional  distribution  of  x(k)  is  a  Gaussian  random  vector,  and  so  is 
parameterized  by  its  conditional  mean,  x.(kjk),  and  conditional  covariance 
£(k)  which  are  given  by  (k  =  0,  1,  — ) 

z(k-s-l)  =  T(k+l)A(k)P(k)z(k)+T(k+l)A(k)V*(k)i(k)+T(k+l)B(k)u(k) 

(5.2.5) 

x(kk)  =  P(k)z(k)+V*(k)£(k) 


:(k  +  1)  =  A(k)  -  V  (k  +  l)C(k  +  l)A(k) 


:(0)  =  :  -  v  (0)C(0)Z 

— o  —  — o 


(5.2.6) 


where 


l(k)  k  A(k)Z (k)_A_*  (k)+R(k)  ;  V*(0)  =  £  G'  (0)  (C(0)£^C'  (0)+Q(0))"1 

(5.2.7) 


and 

V*(k+1)  e  U.  (Z(k))  =  {V  c  M  |  V[C(k+lM(k)C 1 '  (k+l)+fi(k)  ]  =  A(k)C’(k+l)} 
K  '  nm 


P(t),  I(t)  satisfy  the  relation  P(k)T(k)  +  V  (k)C(k)  =  I  . 

_  —  •<  —  —  —  —  — n 

Proof:  Let  us  break  x(k)  into  two  vectors: 

x(k)  =  x^(k)  +  jc  (k)  (5.2.8) 

where  x^(k),  _*2(k)  z  Rn  are  given  by 

25?(k  +  1)  =  A(k)x?(k)  +  £(k)  ;  x9(0)  =  x(0)  (5.2.9) 

:t^(k  +  1)  =  A(k)^(k)  +  B(k)_u(k)  ;  jc.  (0)  =  £  (5.2.10) 

*  V 

and  u(k)  is  of  the  form  (5.2.4).  From  (5.2.10),  we  note  that  {jc.(i) } 
is  F(k,U(0,k  -  1)) -measurable,  and  so  we  have  from  (5.2.8)  that 

x(k|k)  =  E{x,(k)|F(k,U(0,k  -  1))}  +  x^k)  .  (5.2.11) 

Let  us  define 


y.,(k)  =  v(k)  -  C^x^k)  =  C(k)x?(k)  +  _n(k)  (5.2.12) 

and  define  F?(k)  4  F(jy2(0)  , . . .  ,£2(k))  .  ti(i-)  and  are 

If 

F(U(0,k  -  1)  ,k) -measurable,  so  (jy2(i)K_Q  ds  F(k,U(0,k  -  l))-measurable; 
therefore 

F2(k)C  F (k,U(0,k  -  1))  .  (5.2.13) 

Using  (5.2.4),  (5.2,10)  and  (5.2.12)  we  have 

^  (k+1)  =  A(k) ^  (k)+B (k)£(k  ^  (0)+C  (0) (0) , . . .  ^  (k)+C  (k)  j^  (k)  )  ; 

xx(0)  =  0  .  (5.2.14) 

Inductively,  we  have  bc^(i)]^_Q  is  F2(k) -measurable,  and  so  from  (5.2.12), 
y(k)  is  F2(k) -measurable. 


We  have  then 
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F(k,U(0,k  -  1))C  F2(k)  (5.2.15) 

(5.2.13)  and  (5.2.14)  imply  that 

F2(k)  =  F(k,U(0,k  -  1))  .  (5.2.16) 

Let  us  define 

x2(k|k)  =  EU2(k)lF,(k)}  (5.2.17) 

(5.2.11)  and  (5.2.15)  give 

x(k|k)  =  S2(k|k)  +  xx(k)  .  (5.2.18) 

Now  consider  the  stochastic  system  S9  and  the  deterministic  system  ^  described 
by 

x  (k  +  1)  =  A(k)x  (k)  +  C(k)  ;  x  (0)  =  x(0)  G(*  ) 

i.  Z  Z  -  O  - o 

S9:  (5.2.19) 

X2(k)  =  C(k)x9(k)  +  r(k) 

x^(k  +  1)  =  A(k)x(k)  +  B(k)u(k)  ;  >:^(0)  =  £ 

§x:  (5.2.20) 

X1(k)  =  C(k)x2(k) 

Since  x^(0)  is  known  exactly,  21^ (k)  can  be  reconstructed  by  any  class  of 
observers.  The  conditional  distribution  of  (k)  given  F2(k)  is  Gaussian, 
the  conditional  mean,  x;9(k|k),  and  the  conditional  covariance,  £(k) ,  are 
given  by:  (Chapter  3,  section  3.2  and  section  3.3) 

2  z2(k+l)  =  T(k+l)A(k)P(k)z2(k)+T(k+l)A(k)V,!:(k)22(k)  ;  z^O)  =  1(0)^ 

T  x2(kik)  =  P(k)z2(k)+V*(k)_y2(k) 


(5.2.21) 
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and 


E(k+1)  =  A(k)  -  V*(k+l)G(kfl)A(k) 

(5.2.22) 

Z(0)  =  -  Z^C,(0)[C(0)Z^C,(0)  +  0(0)  T^O).^ 

A(k)  k  A(k) E (k)A* (k)  +  R(k)  (5.2.23) 

V*(0)  =  E^  C *  (0)  (C(O)E^C'  W-KKO))"1  ;  V*(k+1)  e  Uk(Z(k))  ;  k  =  0,  1,  ... 

(5.2.24) 

Construct  x^(k)  by  using 

z,  (k+1)  -  T(k+l)A(k)P(k)z1  (k)+T(k+l)A(k+l)V*(k)x1  (k)+T(k+l)B(k)u(k) 

V  is 

x^(k)  =  JP  (k)^i(k)+V  (k)y_2(k)  ;  ^(0)  =  0 

(5.2.25) 

where  {V  (k.) }^_q  is  given  by  (5.2.21)  to  (5.2.24).  From  (5.2.19),  (5.2.20) 
and  (5.2.8),  we  have 


y(k)  =  ^(k)  +  x2(k)  •  (5.2.26) 

Define  the  vector 

^(k)  =  z^(k)  +  z_2(k)  •  (5.2.27) 

By  equations  (5.2.18),  (5.2.21)  to  (5.2.26),  we  have  the  conditional  mean 
estimate  of  x(k)  generated  by 

„  z(k+l)  =  T(k+l)A(k+l)P(k)x(k)+T(k+l)A(k)V#(k)x(k)+T(k+l)B(k)u(k) 
ef,:  *  (5.2.28) 

k^(kjk)  =  P^(k)z^(k)+V  (k)x(k)  ;  _z(0)  =  1(0)3^ 

with  {V  (k) )k_Q  given  by  (5.2.21)  to  (5.2.24). 


Using  equations  (5.2.8)  and  (5.2.18),  we  have 

x(k'k)  -  x(k)  =  x9(kjk)  +  x^k)  -  x^(k)  -  x,(k)  =  x,(k|k)  -  x7(k) 

(5.2.29) 

W 

il(k)},  _  given  by  (5.2.21)  to  (5.2.24)  is  the  conditional  covariance  o£ 
j*2 (k) ,  and  so  it  is  also  the  conditional  covariance  of  x(k) .  Since  x.,  (k) 
is  F(k,U(0,k  -  l))-neasurable,  (5.2.8)  implies  that  the  conditional  dis¬ 
tribution  of  ^c(k)  is  Gaussian,  by  virtue  that  the  conditional  distribution 
of  x. (k)  is  Gaussian. 

We  note  fron  (5.2.3)  and  (5.2.4)  that  the  accumulative  statistic  at 
tine  k  depends  on  the  control  chosen  which  in  turn  depends  on  past 
accumulative  statistics.  But  as  long  as  we  are  interested  in  the  present 
state  of  the  system,  the  information  contained  in  F(k,U(Q,k  -  1))  about 
jc(k)  is  equivalent  in  some  sense  to  the  statistical  information  contained 
in  the  conditional  distribution  of  ^(k) .  Theorem  5.2.1  says  that  the  con¬ 
ditional  distribution  of  x(k)  is  Gaussian,  and  thus  all  the  statistical 
information  revealed  by  accumulated  observation  statistics  is  summarized 
by  the  conditional  mean,  x(k|k),  and  conditional  covariance,  II(k) .  From 
(5.2.21)  to  (5.2.23),  we  see  that  2.(k)  can  be  precomputed  before  any  ob¬ 
servation  is  made  and  any  control  is  applied.  Therefore,  all  the  statistical 
information  about  the  state  at  time  k  is  summarized  in  the  random  vector 
x(k‘k). 

5 . 3  Stochastic  Control  of  Discrete  Linear  Systems  with  Quadratic  Criteria 

In  this  section,  we  shall  consider  the  problem  of  controlling  the 

discrete  linear  system  with  quadratic  criteria: 

N-l 

J(u)  =  eJx’  (N')F  x(N)  +£(x’<k)W(k)x(k)  +  u’(k)M(k)u(k))  J  (5.3.1) 

k~0 
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vith  HO,  W(k)  >  0,  and  M(k)  >  ().  We  are  to  find  a  control  law  of  the 

form  (5.2.4)  which  will  minimize  (5.3.1)  subject  to  (5.2.1). 

Using  lemma  2.2.6  and  (5.2.4),  the  cost  J(u)  can  be  rewritten  as: 

N-l 

J(u)  =  e|e{x' (N)F  x(N) |f(N,U(0,N-1)}  +  ^  E(x' (k)W(k)x(k)+c' (k)M(k)u(k) , 

k=0 

F(k,U(0,k-l) ) )} 

N-l 

=  eJ x *  (N i N) F  x (N ! N)  +  £(*•  (k| k) W (k) x (k \ k)+u  ’  (k)  M(k)  u(k))} 

k=0 

N-l 

+tr(F  Z(N)  W(k)Z(k)J  (5.3.2) 

V  k=0  ' 

where  {Z.(j))j_Q  is  given  by  (5.2.22)  to  (5.2.24).  Since  ( j ) }^_q  is  inde¬ 
pendent  of  the  control,  minimizing  (5.3.1)  is  equivalent  to  minimizing 

N-l 

J'(u)  =  e{x’(N|N)F  x(N|N)  +Ym(  x’  (k | k) W(k) x (k [ k)  +  u'  (k)M(k)u(k))J 

k=0  (5.3.3) 

From  (5.2.5),  the  equation  for  ji(k|k)  is  given  by 

^(k-Mjk+1)  =  A(k)x(kjk)  -  V  (k-H)£(k+l)A(k)  (x(k  |k)  -  3t(k)  >  +  Bi(k);a(k) 

+  V*(k+l)C(k+lK(k)  +  V*(k+l)n(k+l)  (5.3.4) 

& 

where  _V  (k) ,  k  =  0,  1,  ...»  N  are  given  by  (5.2.21)  to  (5.2.23).  The 
process  {x.(k)  )jc_q  is  given  by  (5.2.1).  We  have  now  a  stochastic  problem 
to  solve:  Find  a  control  law  of  the  form  (5.2.4)  such  that  the  cost  (5.3.3) 
is  minimized  subject  to  the  constraints  (5.3,4)  and  (5.3.1). 

Lemma  5.3.1:  The  control  law 

u*(k)  =  -(M(k)+B'(k)K(k+l)B(k)r1B' (k)K(k+l)A(k)S(k|k)  (5.3.5) 

K(k)  =  A(k) (K(k+l)-K(k+l)B(k) (M(k)+B ' (k)K(k+l)B(k) )_1B ' (k)K(k+l) ) 


A(k)+W(k) 


K(N)  =  F 


(5.3.6) 
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Let 

x  (k-rl)  =  A(k)x  -  V(k+l)C(k+l)A(k)(x-x(k))  +  V*(k+l)C(k+lK(k) 

+  V(k+l)n(k)  +  B(k)u*(k)  (5.3.12) 

x°(k+l)  =  A(k)x  -  V(k+l)C(k+l)A(k)(x-x(k))  +  V*(k+l)C(k+l)£(k) 

+  V(k+l)ji(k)  +  ji(k)ii0(k) 


(5.3.13) 
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where  u*(k)  is  given  by  (5.3.5)  with  x  replacing  £(k|k),  and  u°(k)  is 
F(k,U(0,k  -  l))-measurable  function.  We  have  from  (5.3.12),  (5.3.5), 
(5.3.6): 

Etx'WMx+u*'  (k)M(k)u*(k)  | F(k,U(0,k-l)  }  =  *'K(k)x-x'A‘  (k)K(k+l)A(k)x 
(k)B’  (k)K(k+l)A(k)x+u*'(k)M(k)u*(k) 

«  x'K(k)x(k)-x*'  (k+l)K(k+l)x*(k+l)+u*,(k)  (M(k)+B‘  <k)K(k)B(k))u*(k) 

+x'A'  (k)K(k+i)B(k)u* (k)+tr{V(k+l)C(k+l)A(k)K(k+l)} 

-  x,K(k)x(k)-x*,(k+l)K(k+l)x*(k)+tr{(A(k)-E(k+l))K(k+l)}  .  (5.3.14) 


.  Combining  (5.3.9)  and  (5.3.14)  we  have 

<  E{x,W(k)x(k)-ru*'(k)M(k)u*(k)+C(k+l,x*(k+l)|F(k,U(0,k-l))}-C(k,x)  =0  . 

j  ’  (5.3.15) 

Since  u° (k)  is  F(U(0,k-l) ,k) -measurable,  we  have  from  (5.3.13),  (5.3.5), 

1  (5.3.6): 

I 

E{x,W(k)x+uO,(k)M(k)u°(k)|F(k,U(0,k-l))}  =  x'K(k)x-x’A: (k)K(k+l)A(k)x 

+x'A' (k)K(k+l)B0:) (M(k)+B' (k)K(k+l)B(k))_1B' (k)K(k+l)A(k)x+u°' (k)X(k)u°(k) 
1  =  x' K(k) x-x*f(k+l) K(k+1) x° (k+l)+u° '  (k)  (M(k)+B '  (k)K(k+l)B  (k)ju° (k) 

+u0' (k) (M(k)+B' (k)K(k+l)B(k))u*(k)+u*' (k) (M(k)+B* (k)K(k+l)B(k) )u° (k) 

|  +u*(k)  (M(k)+B'  (k)K(k+l)  15  (k) )  u* (k)  +tr { (A(k)-E (k+l)K(k+l) )  } 

|  =  x'K(k)x-x°(k+l)K(k+l)x0(k+l)+(u0(k)-u*(k)) '  (M(k)+B'  (k)K(k+l)B(k))  (u°(k) 

j;  -u*(k))+tr{A(k)-E(k+l)K(k+l)}  .  (5.3.16) 


i 


Combining  (5.3.9), 
K(k+1)  ^  0: 


(5.3.16)  and  (5.3.15)  v/e  have  since  M(k)  >  0  and 
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0  »  E{x'W(k)x+u*' (k)M(k)u*(k)+C(k+l,x*(k+l) | F(k,U(0,k-l)) }-C(k,x) 

<  E{x' W(k)x+u° * (k)M(k)u°(k)+C(k+l,x°(k+l) !F(k,U(0>k-l))}-C(k>x)  . 

(5.3.17) 

The  lemma  follows  from  the  Optimality  Criterion. 

Theorem  5.3.2:  The  control  law  u  (k) ,  k  =  0,  1,  ...,  N  given  by  (5.3.5) 
and  (5.3.6)  is  the  optimal  control  lav/  which  minimizes  the  cost  (5.3.1) 
subject  to  (5.2,1)  and  (5.2.4).  The  optimal  cost  to  go  can  be  expressed 
as: 

N-l 

J*(k,x)  =  x'K(k)x  +  tr^[(A(i)  -  JC  (i+l)}K(i+l)  +  W(i)jL  (i)  ]  +  F  I(N) 

1=k  (5.3.18) 

This  follows  trivially  from  lemma  5.3.1  and  equation  (5.3.2). 

Mote  that  _E(k) ,  A/k) ,  iC  (k) ,  k  =  0,  1,  ...»  M  can  all  be  precomputed 
when  the  noises  distribution  laws  and  the  weightings  (F,  W(k) ,  M(k))  a^e 
all  given.  The  performance  measure  can  be  easily  evaluated  when  the  con¬ 
ditional  mean  of  the  state  vector  is  computed  via  a  minimal  order  optimum 
observer-estimator.  From  (5.2.27)  and  (5.3.5),  we  see  that  the  optimal 
control  law  can  be  written  as 

u* (k)  =  -(M(k)  +  B!(k)K(k+l)B(k))"1B' (k)K(k+l)A(k)F(k)z(k) 

-(M(k)  +  B^k)K(k+l)B(k))“1B'(k)K(k+l)A(k)V*(k)^(k)  .  (5.3.19) 

& 

Denote  the  pure  feedback  portion  of  u  (k)  by 

u*(k)  =  - (M(k)  +  B' (k)K(k+l)B(k))_1B' (k)K(k+l)A(k)VX(k)x(k)  (5.3.20) 
and  feedback  after  compensation 

u*(k)  =  -(M(k)  +  B'  (k)K(k+l)B(k))“V  (k)K(k+l)A(k)P(k)z(k)  .  (5.3.21) 
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The  optimal  control  is  composed  of: 

11  (k)  !i  u^(k)  +  ^2^)  •  (5.3.194) 

The  detail  structure  of  the  optimal  control  system  is  described  in  Figure 
5.1. 

When  the  observation  noise  is  nondegenerate,  i.e.,  (£(k)  >  (), 
k  =  0,  1,  — ,  we  have  the  usual  separation  results  first  derived  by 
Joseph  and  Tou.  Theorem  5.3.2  indicates  that  separation  is  true  under 
more  general  assumptions  when  £00  and  R(k)  are  nonnegative  definite  and 
even  when  they  are  both  zero  matrices.  The  theorem  can  also  be  applied 
to  the  case  when  the  observation  noise  is  sequentially  correlated.  In  the 
following,  we  shall  treat  this  special  case  in  some  detail. 

Consider  the  system  described  by 

x(k  f  1)  =  A(k)x(k)  +  jB(k)ii(k)  +  j^(k) 

§2:  (5.3.22) 

Z(k)  =  C(k)x(k)  +  n(k) 

CD 

{jl(k)}^_Q  is  sequentially  correlated  ar.d  is  described  by 

n(k  +  1)  =  A(k)n(k)  +  X00  •  (5.3.23) 

We  shall  assume  that  jj.(k) ,  X(k) ,  k  =  0,  1,  ...,  x(0)  and  ji(0)  are  inde¬ 
pendent  Gaussian  random  vectors  with  statistical  laws  given  by  (3.3.2), 
(3.3.3)  and  (3.7.14).  The  control  problem  is  to  find  control  u  (k)  of 
the  form  (5.2.4)  which  will  minimize  the  cost  (5.3.1)  subject  to  (5.3.22), 
(5.3.23).  From  (5.3.22)  and  (5.3.23)  we  have  the  augmented  system 
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(5.3.24) 


x°(k  1)  =  A  (k)x  (k)  -5-  3  (k)u(k)  -f  r(k) 


*a! 


va(k)  =  Ca(k)xa(k) 


•-here 


a..  % 
x  (k)  = 


X(k)' 

;  Aa(k)  = 

A(k)  :  0 

;  3a(k)  = 

B(k) 

^(k) 

o  :  A(k) 

L~  •  “  J 

0 

-;a(k)  = 


I(k) 

r(k) 


Ca(k)  =  [C(k)  I  I  ] 
—  —  .  — n 


tne  cost  (5.3.1)  can  be  written  as 

s-i 


(5.3.25) 


Ja (u)  =  e{ xa •  CO  rV CO  (k)Ka (k)xa (k)  +  u  ’ <k)X(k)u(k))J  (5.3.26) 


k=0 


wnere 


.-a 


(5.3.27) 


The  augmented  control  problem  is  to  find  ii(k) ,  of  the  form  (5.2.4) 
such  that  the  augmented  cost  (5.3.26)  is  minimized  subject  to  the  aug¬ 
mented  sys.en  (5.3.25)  and  constraint  (5.2.4).  Ive  note  that  the  solu¬ 
tion  for  the  augmented  control  problem  is  the  same  as  that  of  the  original 
control  problem. 

Apply  theorem  5.2.3  to  the  augmented  control  problem,  we  have 


u  (k)  =  -  (M(k)  +  B 


a. 


(k)Ka(k+l)Ba(k))  1Ba' (k)Ka(k+l)Aa(k)xa(k|k) 


(5.3.28) 
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*(k)  =  A  ,(k)(Ka(k+l)-Ka(k+l)Ba(k)(M(k)+Ba,(k)K(k+l)B3(k))"1Ba,(k) 


Ka(k+l)Aa(k)+Wa(k)  ;  Ka(N)  =  Fa  (5.3.29) 


and  x  (k;k)  is  given  by: 

z(k+l)  =  T(k-K)Aa(k)P(k)z(k)  +  T(k+l)Aa(k) V*(k)y(k)  +  T(k+l)Ba(k)u*(k) 

ea: 

^  a  *  a 

xa(kjk)  =  P(k)z(k)  +  V  (k)v(k)  ;  z(0)  =  T(0)>£ 

(5.3.3C 


V~(0)  = 


zj;'  (0) 

(C(O)I  C' (0)  +  zn}_1 

—  — Q —  — o 

_n 


(5.3.31) 


Za(k  +  1)  -  £,a(k)  -  V(k  +  i)Ca(k  +  l)Za(k) 


(5.3.32) 


:a(k)  =  Aa(k):a(k)Aa,(k)  +  Ra(k)  ;  V“(k  +  1)  s  U(:a(k)) 


(5.3.33) 


Lemma  5.3.3:  The  solution  of  (5.3.29)  is  given  by 


K(k)  :  0 

Ka(k)  =  ...  '  .. 

o  :  o 


(5.3.34) 


with  K(k)  given  by  (5.3.6). 

Proof:  We  shall  use  the  induction  method.  At  k  =  N’,  (5.3.27),  (5.3.6), 
and  (5.3.29)  give 


F  .*  0  K(N)  :  0 

l'Q/Vl  =  •  - 

J  •••  •••  •••  ••• 

o  :  o  o  :  o 


(5.3.35) 


Assume  chat  cha  statement  is  true  at  k  +  1,  we  have  from  (5.3.25), 


(5.3.27),  and  (5.3.29)  that 


K2(k)  = 


A(k)  (K(k+l)-K(k+l)B(k)  (M(k)+B ’  (k)K(k+l)B(k) )  V  (k)K(k+l) }A(k)+W(k)  *.0 

*  •  •  • 

0  0 


'K(t)  :  o 

o  :  o 


(5.3.36) 


and  so  the  lemma  follows. 

Theorem  5.3.4:  Tin  control  law, 


u*(k)  =  -(M(k)  +  B' (k)K(k+l)B(k))“1B,(k)K(k+l)A(k)x(k|k)  (5.3.37) 


with  K(k)  given  by  (5.3.6)  and 

x(kjk)  =  [I  I  0  )xa(k|k)  (5.3.38) 

is  the  optimal  control  law  which  minimizes  the  cost  (5.3.1)  subject  to 

(5.3.22),  (5.3.23),  and  (5.2.4).  The  optimal  cost  to  go  is 

N-l 

J*(k,x)  =  x'K(k)x+tr^[(Aa(i)-Za(i+l))Ka(i+l))+Wa(i)Za(i)]+FaZa(N) 

i=k  (5.3.39) 

This  follows  easily  from  theorem  5.3.2,  lemma  5.3.3,  and  equation  (5.3.28). 

Note  chat  >;a(k)  t  Rn+m,  and  so  jz(k)  z  F.n  (see  chapter  3).  The  detail 
structure  of  optimal  control  system  is  described  in  detail  in  Figure  5.2. 

5 . 4  Estimation  with  Feedback  for  Continuous  Linear  Systems 
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x(t)  ~  A(t)x(t)  +  B(t)u(t)  +  ^(t) 


iu. 

where  x(0  c  Rn,  u(t)  e  Rf,  S,(t)  r.  RP,  j}(t)  t  R  ,  £<t)  t 


(5.4.1) 


We  assume 


that  x(t  ),  U(t),  tit},  (n(t),  tit}  are  independent  statistics. 
2c(tQ)  GO^,^)  and  _C(t),  ji(t) ,  t  i  t  are  white  Gaussian  noises  with 
properties  (4.3.2).  The  control  u(t)  is  feedback  in  nature. 


Let  us  denote  U[0,t)  =  (ui(t):t  e  [0,t)},  and 
Yu[0  cj[0,t]  =  (iyfQ  t)(T):T  f-  [0,t]}.  The  observation  statistic  at  time 
C  is  ~U[0,t  ^(t)  (  the  subscript  is  to  indicate  that  the  statistic  is  de¬ 
pendent  on  the  past  control  values).  The  accumulative  observation  statistic 


at  time  t  is  tj[0,t].  We  shall  assume  that  the  control  at  time  t  is 

a  function  of  accumulative  observation  statistic: 


u(t)  =  Kt.Yyj  jlO.tl)  .  (5.4.2) 

Denote  F(t,U[0,t))  =  ^(Y^^q  tj[0,t]).  The  control  jj(t)  is  a  random  vector 
which  is  F(t,U[0,t))-measurable. 

Let  _f(s)  be  continuous  on  [0,t]  with  values  in  Rm,  define  the  exten¬ 
sion  of  J:(s)  by 


!_f  (s)  0  £  s  <  t 

(5.4.3) 

f(t)  t  i  s  <  T 


17 chus  defined  is  in  Cm[0,T],  the  class  of  continuous  function  defined 
on  [0,TJ  with  values  in  R  .  The  control  (5.4.2)  can  be  expressed  as 


1 
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»<c>  -i<t.-JBIOit;)rO,tl)  (5.4.4) 

where  i_(*,*)  is  viewed  as  a  napping  from  R  <  C  {0,T]  -»  Rr.  The  control 

c 

(5-4.4)  is  also  r(t,U[0,c)) -measurable.  U’e  assume  that  c^(t,-)  satisfies 
a  Lipschitz  condition: 

r  3(t»f)  -  i(t,g)  ‘  <  a  ! if  -  gf  ?  ;  f,g  €  c  [0,T)  (5.4.5) 

Q 

tor  all  t  •'  (0,T]  where  is  some  constant.  Sometimes  we  shall  subpress 
the  dependence  on  past  control  value,  and  write  (5.4.4)  as 


u(t)  =  i.(t»~cy)  ;  y  s  {y( t)::  £  fo,t]> 


(5.4.6) 


without  causing  confusion. 

Theorea  5.4.1:  Let  £2(t)R(t)C' (t)  >  0,  and  the  control  is  of  rhe  feedback 
fora  (5.4.6).  The  conditional  distribution  of  the  current  state  of  is 
Gaussian  random  vector,  and  is  parameterized  by  the  conditional  mean, 
x(t  t),  and  com"  “icnal  covariance,  £(t),  which  are  given  by: 

e^C(LX):  z(t)  =  (T(t)A(t)P(cHT(t)P(£)-T(t)L*(t)C1(t)P(t))z(t)+T(t)L*(t)v1(t) 
-f(I(t)A(t)V*(tHT(t)V2(c)-T(t)L*(t)C1(t)V2(t))x2(t) 

+r(t)B(t)u(t) 

i(to>  =  IfVVl(co)4(tO+,i2(to) 


(5.4.7) 


x(c  t)  =  P(t)_z^t)+V?(t)jjr-(t) 


-1 


:(t)  =  (A(t)-R(t)Cj(t)£  ^ (t)£2 (t) )£(t)+Z (t)  (A(t)-R( t)C^2 ( t)^"”1  (t)C^ (t) )  ' 

-:(t)(c^(c)i"1(t)c2(tHc’(t)£"1(t)c1(t)):(t)+R(t) 


-1, 


-R(t)C’(t)4  (t)C2(t)R(t) 

-iD-i0£2<t0)(£2(to)4£2(y,'1£2(to)4  ;  t(t)  S  C2(t)R(t)C'(t) 


(5.4.8) 
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Let  F?(t)  =  F(v2(t),t  e  (0,t)) .  Since  •  v(s)  .^(s)  are  F(t ,U[0, t) )- 
measurable,  then  tv.9(s)}^SSQ  is  F(t,U[0,t))-measurable  and  so 

F,(t)  C  F(t,u[0,t))  .  (5. A. 17) 

From  (5.4.16)  and  (5.4.13),  we  have 

£(t)  =  £9(t)  +  C(t)  [  ^A(t,i)B(T)u(T)dT  (5.4.18) 

J  t 

o 

where  u(t)  is  of  the  form  (5.4.6).  Equation  (5.4.18)  is  an  integral  equa¬ 
tion.  By  the  Lipschitz  assumption,  equation  (5.4.18)  can  be  solved  by 

successive  aporoximations  to  yield  a  unique  y  z  C  [0,T].^  Setting 

m 

v(o)(t)  =  0  and 

X(v)(t)  =  I9(t)+C(t)ft  c,(t,T)B(T)6(T,7t.y(y"1))dT 

•  t  1 

o 

t  £  [0,T]  ;  V  =  l,  2 . (5.4.19) 

Inductively,  *_£  '^(s)}^_q  is  F^O-measurcd^e  for  \>  =  1,  2,  . ..;  and  so 
'•X(s);=  lim  (s) -^_q  is  also  F?  (t)-measurable ,  and 

F2(t)  3  F(t,U[0,t))  .  (5.4.20) 

Combining  (5.4.17)  and  (5.4.20),  we  have 

F2(t)  =  F(t ,U(0, t) )  .  (5.4.21) 

Equation  (5.4.15)  becomes 

x(tjt)  =  ( t: )  +  B{x2(t)  |  F?(t) ;  .  (5.4.22) 


Mow  consider  the  systems: 
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x,  (t)  =  A(t)x  (t)  +  B(t)u(t)  ;  x. (t  )  =0 

x  X  X  o 

S^:  (5.4.23) 

^(t)  =  C(c)x1(t) 


x,(t)  =  A(t)x,(t)  +  J,(t)  ;  =  'v 


gc. 


— o  ~0 


(5.4.24) 


>:o<c)  = 


^t(t)x2(t)  +  n.(t) 


C2(t)x2(t)  J 


Q 

Apply  observers  theory  to  the  deterministic  system  and  stochastic 
systems  §2  (see  chapter  4,  sections  4.2,  4.3).  In  this  manner  we  prove 
the  theorem  easily.  For  detail  procedures  the  reader  is  referred  to  theorem 
5.2.1,  where  we  have  proved  the  discrete  analog  in  great  details. 

5 . 5  Stochastic  Control  of  Continuous  Linear  Systems  with  Quadratic  Criteria 

Q 

We  consider  the  problem  of  controlling  the  continuous  linear  system 
with  quadratic  criteria 


jC(u)  =  E{x'(T)F  x(T)  + 


fT 

x’ (t)W(t)x(t) 

t 


+  u'  (t)M(t)u(t)dtj 


(5.5.1) 


o 

with  £  -  0,  W(t)  i  J)  and  M(t)  >  We  are  to  find  a  control  of  the  form 
(5.4.4)  and  (5.4.5)  such  that  (5.5.1)  is  minimized  subject  to  (5.4.1). 

For  any  control  of  the  form  (5.4.4)  and  (5.4.5)  we  have  from  lemma 
2.2.6  that 


\ 


=  E{x'  (TiT)F ;  x(T|T)+.  x'(c  t)H(t)x(tit)V(c)M(t)u(t)dc| 

*  c 
fT 

+tr{F  ;.nT!T)+!  W(t)L_(t)dt| 

*  t 


(5.5.2) 


where  I(t),  t  i  t  ,  is  given  by  (5. A. 8).  We  note  from  (5.4.8)  that  £(t) 
o 

is  independent  of  the  control  function;  thus  to  minimize  (5.5.1)  is  equiva¬ 
lent  to  minimizing 

i-T 

JC,(u)  =  Ejx'diDF  x(TjT)  +  |  x' (t|t)W(t)x(t|t)+u’ (t)M(t)u(t)dt}  .  (5.5.3) 

•  t 

o 

From  (5.4.7),  we  can  easily  derive  the  differential  equation  for  x(tjt): 
x(t|  t)  =  A(t)x(t  I  t)+(V2(t)C2(t)-h^(t)C1(t))  (x(t '  t)-x(t))+V9(t)C9(tK(t) 


+Li(t)jn(t)+B(t)u(t) 


(5.5.4) 


with  C9(t)  =  £9(t)  +  C9(t)A(t),  and  x(t),  t  -  tQ,  is  a  diffusion  process 
given  by  (5.4.1).  We  have  now  a  stochastic  control  problem:  Find  a  control 
law  of  the  form  (5.4.4)  and  (5.4.5),  such  that  the  cost  (5.5.3)  is  minimized 


subject  to  the  constraints  (5.5.4),  (5.4.1). 
Lemma  5.5.1:  The  control  law 


u  (t)  =  -M  (t)B' (t)K(t)x(t  t) 


(5.5.5) 


-K(t)  =  A’(t)K(t)+K(t)A(t)-K(t)B(t)H_X(t)S!(t)K(t.)+W(t)  ;  K(T)  =  F 


(5.5.6) 


is  the  optimal  law  for  the  above  stochastic  control  problem,  i.e.,  if 
u°(t)  is  a  control  of  the  form  (5.4.4)  and  (5.4.5),  then 


JC,(u':)  :  JC,(u°) 


(5.5.7) 
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f1  *. 


The  optimal  cosfto-go  is 

J^’(t,x)  =  e{x*’(T|T)F  x*(TiT)+|  £'  (-)'A(-'ju' (-)d-. 

x^Ctit)  =  X  } 

•T 

=  x’K(t)x+tr  J  (V*(t)C2(t)R(t)C^(t)V*’(t) 

+Lj(T)^(T)L,J’(T))K(T)dT  .  (5.5.8) 


Proof :  As  in  the  discrete  analog,  we  shall  make  use  of  the  Optimality 
Criterion  (theorem  2. A. 4)  to  prove  the  lemma.  Let  us  define  for  all 
(t,x)  t  [0,T]  x  Rn: 

*T 

C(t,x)  =  x'K(t)x+tr|J  (V2(T)C2(-)R(T)C*(T)V**(T)+Lj(T)2(T)L*,(T))K(T)d-| 

(5.5.9) 

where  K(t)  satisfies  (5.5.6).  From  (5.5.6)  and  (5.5.9),  we  have 


C(T,x)  =  x'F  x  .  (5.5.10) 

Let  U[0,t)  be  an  arbitrary  control  function  and  denote 

x  =  E{x(t)|F(t,U[0,t))}  =  x(t|t)  .  (5.5.11) 

Let  u  (t)  be  given  by  (5.5.5)  with  x(t|t)  replaced  by  x.  Denote  the  dif¬ 
ferential  generator  of  x(t|t)  by£  (•),  we  have  from  (5. 5. A),  (5.5.5), 
(5.5.6): 

E {£u*  (C  ( t  ,  k)  )+k ' W ( t ) k+u* '  ( t)M(t )  u*  ( t )  1 F ( t ,  U  [0 ,  t )  )  } 

=  tr{(V2(t)C2(t)R(t)C^(t)V2'(t)+L*(t)2(t)L*(t)iK(t)}+2k’A' (t)K(t)k+ 

2u  ' (t)B' (t)K(t)k+x'W(t)x+u  ' (t)M(t)u  (t) 

=  tr{(V2(t)C2(t)R(t)C^(t)V2(t)+L*(t)2(t)L^' (t))K(t)}-k  K(t)x  .  (5.5.12) 


< 


—  lbb— 


Combining  (5-5-9)  and  <5.5.12)  we  have 


Cc(c,x)+£-.£u*(C(c,xH^,«(r)xru  ’(t)M(t)u(t)  :F(t,ClOac»  •  =  0  .  (5.5.13) 

Lee  u°(c)  be  any  F(t,UlO,t))-me3Surable  function,  we  have  from  (5.5.4), 
(5.5.5)  and  (5.5.6) 


£'£u0(C(c,s)-rx,}C(t>»fnO(c)M(c)u,(t)  F(t,U[0,c))* 

=  c^*:(V*(t)C0(t)R(t)c;(c)V*,  (c)+Lj(t)5(t)L**  (c»K(t)}-i/&(c)*(c) 


+(u°(c)-u~(t))  'M(t)  (u°(c)-u~(t)) 


(5.5.14) 


Since  M(t)  ->  0,  (5.5.13)  and  (5-5.14)  imply 


0  =  Cc(t,x)+E-;£u*(C(t,x)Hx’W(t)x4-u  '(t)M(t)u  (t)  F(t,U[0,t) )  • 

iC  (c  x)-rE<£  0(C(c,x)HxsV<t)x4-u0,(c)X(c)u°(t)  F(t,i'tO,t) ) .  .  (5.5.15) 


The  lemma  follows  from  (5.5.10),  (5.5.15),  (5.5.9)  and  the  Optimality 
Criterion  (theorem  2.4.4). 

From  lemma  5.5.1  and  equation  (5.5.2),  we  have  easily  the  following: 

A 

Theorem  3-5.2:  The  control  law  u  (t)  given  by  (5.5.5)  and  <5.5.6)  j's  the 
optimal,  control  law  which  minimizes  the  cost  (5.5.1)  subject  to  the  con¬ 
straints  (5.4.1),  (5.4.4)  and  (5.4.5).  The  optimal  costrto-go  can  be  ex¬ 
pressed  as 


J*(t,x)  =  x'K(t)x+tr  F  I(T)+  [W(:)Z(i)rV:,(r)C?(:)R(T)C^(-)V2’  (*)K(:) 


+L1(T)2(T)L1(T)K(r)]dT 


(5.5.16) 
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vhere  ._(-),  V.>(-),  L^(t)  are  given  by  (5.4.8)  to  (5.4.10),  and  K(t)  is 
given  by  (5.5.6). 

c 

me  structure  of  the  optimal  control  system  for  is  described  in 
Figure  5.3,  where  we  have  decomposed  the  control  law  into 

u  (t)  =  u^t)  +  u2(t)  (5.5.17) 

JL 

u^(t)  is  the  pure  feedback  from  the  noise-free  observation: 

u*(t)  =  -M"1(t)B’(t)K(t)vJ(t)v7(t)  (5.5.18) 

.v 

and  u,(t)  is  a  feedback  after  compensation: 

aj(t)  "  -U’^tJJ'ttJKtOPft^c)  .  (5.5.19) 

In  the  special  case  when  £0(t)  =  0,  i.e.,  all  observation  is  noisy,  we 
have  the  usual  separation  results  due  to  Wonham  [27]. 

The  general  results  can  be  applied  to  the  case  where  *we  have  time- 
correlated  observation  noise. 

Q 

Consider  the  system  described  by  (4.5.52),  the  statistical  law  of 

underlying  certainties  are  given  by  (4.5.53)  and  (4.5.34).  From  these 

c 

assumptions  we  can  form  the  augmented  system  §  given  by  (4.5.55)- 

3 

(4.5.57).  Let  us  define 


We  form  the  augmented  cost 

fT 

/(u)  =  E{xa,(T)FcV(T)+|  xa'(t)W3(t)xa(t)+u'(t)M(t)u(t)it} 

•  t 

o 


.  (5.4.21) 
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The  augmented  control  problem  is  to  find  a  control  of  the  form  (5.4.4), 
(5.4.5)  which  will  minimize  (5.4.21)  subject  to  the  constraint  (4.3.53). 
Mote  that  the  solution  for  the  augmented  control  problem  is  the  sane  solu¬ 
tion  for  the  original  control  problem  where  we  are  to  find  control  of 
form  (5.4.4),  (5.4.5)  so  as  to  minimize  (5.5.1)  subject  to  the  dynamical 
system 

Apply  theorem  5.5.2  to  the  augmented  control  problem,  we  have  the 
optimal  control  law  given  by 


u*(t)  =  -  >f1(t)Ba'(t}Ka(c)xa(t!t) 


(5.4.22) 


-Ka(t)  =  Aa'(t)Ka(t)+Ka(r.)Aa(t)-Ka(t)Ba(t)M  1(t)Ba’ (t)Ka(t)+Wa(t) 

Ka(T)  =  Fa 


(5.4.23) 


3  3 

and  x  (t  t)  the  conditional  mean  of  x  (t),  and  is  generated  via  a  minimal 
order  optimum  observer-estimator  (see  theorem  5.4.1). 

Lemma  5.5.3:  The  solution  of  (5.4.23)  is 


Ka(t) 


K(t) 

0 


0 

0 


(5.4.24) 


with  K(t)  satisfying  (5.5.6). 
Proof:  Partition  Ka(t)  into 


Ka(t)  = 


Ku(t)  :  K12(t) 


-21(t)  I  — 22 (t) 


(5.4.25) 


(5.4.23)  gives: 


-Kn(c)  =  A’ (c)K11(t)+K.11(c)A(c)-Kli(c)B(c)M 

-I1(T)  = 

(5.4.26)  _ki2(c) 

— 12(T)  " 

-K92(C)  =  A'(t;)K„(t)+K„(t)A(c)-K21(t)B(c)M' 

K2o^T>  = 

-K,1(c)  =  -K|,(t) 

Comparing  with  (5.5.6),  we  see  Chat 

KU(C)  =  K(t) 

From  the  second  equation  of  (5.4.26),  we  deduce 

K1?(t)  =  0 

substituting  (5.4.28)  into  the  third  equation  of  (5.4. 


■~22(C)  =  - 

Combining  (5.4.25)  to  (5.4.28),  we  have  (5.4.24). 

Using  Jemma  5.5.3,  theorem  5.5.2  and  equation  (5 
results : 

Theorem  5.5.4:  The  control  law 

u"(t)  =  -M  1(t)B' (t)K(t)x(tjt) 
with  K(t)  satisfying  (5.5.6)  and 


~(t)B'(t)Ku(t)+W(t) 

F 

1(t)B’(t)K1,(t)  ; 

0 

1(t)B(t)K1?(t)  ; 

0 

(5.4.27) 

(5.4.28) 

26)  and  then  we  have 

(5.4.29) 

4.22),  we  have  the 

(5.4.30) 
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x(t|t)  =  [I  I  0]xa(tjt)  =  E{x(t):F(t,C  [  0 ,  t  ] )  ;  (5.4.31) 

a  * 

is  the  optimal  control  law  of  the  form  (5.4.4),  (5.4.5)  which  minimizes 

-  c 

(5.5.1)  subject  to  the  dynamical  constraints  (4.5.28).  (See  Fig.  5.4.) 
5.6  General  Discussions 

In  this  chapter,  we  considered  the  problem  of  controlling  a  linear 
system  with  quadratic  criteria  under  the  assumptions  that 

1)  System  dynamics  are  known, 

2)  Statistical  laws  of  underlying  uncertainties  are  known. 

It  has  been  shown  tr. -tt  under  fairly  general  assumptions  on  the  noise 
structures,  the  optimal  control  strategy  can  be  split  into  two  distinct 
procedures: 

1)  Find  the  conditional  mean  estimates  of  the  current  state 

2)  optimally  feedback  as  if  the  conditional  mean  estimate 

of  rhe  current  state  is  the  true  state  of  the  system. 

(3’] 

This  result  is  generally  referred  to  as  Separation  Theorem'  or  Certainty- 

[431 

equivalence  principle.  *  Theorem  5.3.2  includes  as  special  case  the  re¬ 
sults  obtained  by  Joseph  and  Tou,  Gunckel  and  Franklin;  and  theorem 

[271 

5.5.1  generalized  that  of  Wonham's.  In  the  following,  we  shall  discuss 

some  further  extensions  of  the  research  related  to  this  chapter. 

(A)  Different  Cost  Criteria 

In  this  chapter,  we  have  considered  exclusively  quadratic  criteria. 

The  first  reason  for  doing  this  is  motivated  by  the  perturbation  guidance 

1431 

approach  to  many  guidance  control,  problem,  where  we  try  to  keep  a 
stochastic  system  on  a  precomputed  nominal  trajectorv.  Such  an  approach 
will  naturally  lead  to  the  problem  of  controlling  a  time-varying  linear 
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systemw'ch  quadratic  criteria.  There  is  also  the  reason  that  control 
with  quadratic  criteria  is  one  special  case  where  we  can  derive  explicit 
results. 

[59] 

The  approach  taken  in  this  chapter  follows  that  of  Streibel  in 

r  32  ] 

the  discrete  case,  and  that  of  Wonhamv  in  the  continuous  version. 
Theoretically,  we  can  easily  extend  sections  5.3  and  5.5  to  more  general 
situations  where  the  cost  criteria  is  not  necessary  quadratic.  The  main 
difficulty  that  we  shall  feet  is  the  existence  problem,  which  is  a  mathe¬ 
matical  rather  than  conceptual  issue.  In  general,  we  shall  have  to  formu¬ 
late  and  solve  a  new  stochastic  control  problem  where  the  process  being 
controlled  is  the  "estimated"  process  x(t|t),  rather  than  the  process  x(t). 
The  interested  readers  ara  referred  to  Streibel and  ,.%,onham^J  for  de¬ 
tail  discussions. 

(B)  Terminal  Time  N  •»  °°(T  ») 

In  the  discrete  case,  let  us  define  K(k,N;F)  as: 

K(k,N;F)  =  A'  (k)  (K(k+l,N;F)-K(k+l,N;F)B(k)  (M(k)+B'  (k)K(k+l,N;F)B(k))_1- 

B' (k)K(k+l,N;F)A(k}+K(k)  ;  K(N,N;F)  =  F  .  (5.6.1) 

From  the  separation  results,  the  overall  control  system  can  be  studied 
separately  by  first  considering  the  minimal  order  optimum  observer- 
estimator,  and  then  the  feedback  control.  In  the  case  when  N  -►  «,  the 
error  covariance  will  remain  bounded  if  and  only  if  the  system  §?  is  de¬ 
tectable  (see  chapter  3) .  Thus  detectability  is  necessary  in  order  we 
can  reasonably  talk  about  controlling  the  system  during  an  infinite  time 
span.  Next,  we  have  to  consider  under  what  appropriate  conditions  the 
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feedback  gain  will  remain  bounded.  We  note  chat  this  is  equivalent  to 
consider  under  what  assumptions  will  K(k,N;F)  remain  bounded  as  S  *  ®. 
Comparing  (5.6.1)  with  (2.5.15)  where  we  replace 


A(N-k+k  )  -  A'(k) 
—  o  — 


0  -  ^(k) 


W(N-k+kQ)  -  fi2(k) 


M(N-k+k  )  -  R(k) 
—  o  — 


B(N-k+k  )  -  A* (k)D' (k) 


K(N-k+ko,N;F)  —  P(k,k^;F) 


(5.6.2) 


We  can  view  K(k,N;£)  as  the  minimal  sequence  with  respect  to  a  certain 
solution  sec.  This  allows  us  to  consider  the  asymptotic  behavior  of 
K(k,N;i_)  as  N  »  *.  From  section  4.6,  we  see  that  a  necessary  and  suf- 
ricient  condition  for  lim  K(k,X;_F)  to  remain  bounded  and  satisfy  a  steady- 
state  difference  equation  is  that  there  exists  some  matrix  G(k) , 
k  =  ...,  -1,  0,  1,  ...  such  that 


< 1 » J  > 


i-J 


-  'tie 


(5.6.3) 


where 


A(k)  =  A(k)-B(k)G(k)  ;  J^O'.j)  »  A(i)A(i-l)...A(j)  .  (5.6.4) 

Note  that  (5.6.3)  and  (5.6.4)  are  equivalent  to  saying  that  there  exists 
G(k)  such  that  if  we  use  the  control 
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^j(k)  =  -G(k)x(k) 


(5.6.5) 


The  resulting  system 

x(k  +  1)  =  (A(k)  -  B(k)G(k))x(k)  +  £(k)  (5.6.6) 

is  uniformly  asymptotically  stable.  We  shall  call  such  a  system 
stabilizable.  Thus  detectability  and  stabilizability  are  necessary  and 
sufficient  conditions  which  allow  us  to  consider  control  of  discrete 
linear  system  over  an  infinite  time  span. 

In  the  continuous  time  case,  let  us  define  K(t,T;F)  as  the  solution 
of 

-K(t,T;F)  =  A' (t)K(t,T;F)+K(t,T;F)A(t)-K(t,T;F)B(t)M_1(t)B' (t)* 

K(t,T;F)+W(t)  ;  K(t,T;F)  =  F  .  (5.6.7) 

In  order  that  we  can  consider  the  problem  of  controlling  the  continuous 
c 

linear  system  during  an  infinite  time  span,  first  we  have  to  require 

that  the  error  covariance  will  remain  bounded  as  T  >•  05 .  A  sufficient  con- 

c 

dition  for  this  is  detectability  of  the  system  Sy  Next,  we  have  to  con¬ 
sider  the  asymptotic  behavior  of  K(t,T;£)  as  T  ->  ®,  Comparing  (5.6.7) 
with  (4.3.29)  where  we  replace 


A’(-t)  - 

A(t) 

0  — 

£2(0 

B'(-t)  - 

cx(t) 

M(-t)  - 

S(t) 

W(-t)  - 

R(t) 

K(-t,T;F)  - 

* 

I  (t) 

(5.6.8) 


-1 76- 


We  have  from  theorem  4.4.4  that  lim  K(t,T;F)  will  remain  hounded  if  and 
oniy  if  there  exists  a  G(t)  such  that  (A(t)  -  B/t)G(t))  is  exponentially 
stable.  This  is  equivalent  to  the  condition  that  there  exists  a  feedback 
control 


u(c)  =  -G(t)x(t) 
such  that  the  resulting  system 

x(t)  =  (A(t)  -  B(t)G(t))x(t)  +  i(t) 


(5.6.9) 


(5.6.10) 


will  be  uniformly  asymptotically  stable.  We  shall  call  such  a  system 
stabilizable .  Therefore,  in  the  continuous  case,  detectability  and 
stabilizability  are  sufficient  conditions  which  allow  us  to  consider  con¬ 
trol  of  continuous  linear  system  over  an  infinite  time  span. 

With  the  assumptions  on  detectability  and  stabilizability,  the  asymp¬ 
totic  optimal  cost  rate  is  (see  (5.3.18)) 

\'-l 

lim  "  *  ^  J^(k,x)  =  lim  ^-—7  tr£^(4(i)  -  ^(i+l)K(i-rl)  +  W(i)!_(i)) 

1_k  (5.6.11) 

in  the  discrete  case,  and  (see  (5.4.16)) 


lim 


T 


1 

T-t 


J*(C,x) 


=  lim  tr  1  [W(T):(T)fvJ(T)C2(T)R(T)C^(T)V2,(t)K(T/ 

X-'/.  -  t 

+L*(T)2(T)L*’(T)K(T)}dr  (5.6.12) 


in  the  continuous  case.  We  note  that  the  asymptotic  optimal  cost  rate  is 
independent  of  x. 

In  the  time  invariant  case,  detectability  and  stabilizability  imply 


(see  chapters  3  and  4) 
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1)  We  have  a  Cirae  invariant,  minimal  order,  optimum  observer- 
estimator  which  generates  the  conditional  mean  estimate 
of  the  current  state. 

2)  Wa  have  a  constant  feedback  gain. 

Therefore,  the  optimum  control  system  is  also  time  invariant  whe re  one 
can  write  transfer  functions  for  it. 

The  study  on  the  stochastic  stability  is  a  topic  for  further  research. 
5 . 7  Perspective 

The  Separation  Theorem,  or  certainty-equivalence  principle,  was 

f  56 1 

stated  for  discrete  linear  systems  by  Joseph  and  Tou,  Gunckel  and 

Franklin,  Screibel,^^  and  for  continuous  linear  systems  by 
[221  [27] 

Wonham.  ’  The  assumption  was  that  the  observation  noise  is  non¬ 

degenerate  white  Gaussian  process. 

The  consideration  in  [56],  [58],  and  [27]  is  that  of  quadratic 
criteria  and  the  approach  is  straightforward  application  of  the  Optimality 

[59]  f’2] 

Criterion.  The  investigations  by  Streibel  and  Wonham  include  more 

general  cost  criteria;  the  approach  taken  is  that  of  first  finding  an 

equation  for  the  conditional  mean  of  ••he  current  state,  and  then  formulate 

a  new  optimal  control  problem  where  the  process  being  controlled  is  the 

conditional  mean  process;  finally,  appeal  to  Optimality  Criterion. 

f?2] 

The  approach  taken  in  this  chapter  is  that  of  Won'nam's.  .  The  cost 
criteria  we  considered  is  quadratic,  but  one  can  easily  extend  the  results 
to  more  general  cost  criteria.  The  assumption  that  the  observation  noise 
is  a  nondegenerate  white  Gaussian  process  was  relaxed.  It  was  proved 
that  Separation  holds  when  the  observation  noise  is  one  of  the 
following: 

1)  regular  white  Gaussian  process 
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2)  degenerate  white  Gaussian  process 

3)  totally  singular  situation  (i.e.,  noise-free  observation) 

£)  colored  noise  (i.e.,  sequentially  correlated  or  time- 

correlated) 

5)  summation  of  colored  and  white  Gaussian  noise. 


CHAPTER  VI 


CONTROL  OF  DISCRETE  TIME  LINEAR  SYSTEMS  WITH 
UNKNOWN  GAIN  PARAMETERS 


6-1  Introduction 

We  have  considered  the  control  of  linear  sy scans  with  unknown  dynamics 
in  the  last  chapter.  Now  we  shall  relax  sone  of  the  assunptions  that  all 
dynamics  are  known.  In  many  practical  control  problems,  we  are  confronted 
with  the  problem  cf  controlling  an  unknown  linear  svsten.  We  nay  have  a 
crude  idea  about  the  dimension  of  the  system  but  the  zero  and  pole  loca¬ 
tions  may  not  be  fully  known.  In  this  chapter  we  shall  consider  linear 
systems  whose  coles  are  known  but  whose  zeroes  are  unknown.  We  shall 
generalize  this  to  the  case  of  a  dynamical  system  in  which  the  gain  vector 
is  unknown.  Admittedly,  the  situation  in  which  we  are  to  control  a  linear 
system  with  unknown  gain  is  rare;  however,  this  research  effort  is  neces¬ 
sary  and  of  importance  in  guiding  our  way  to  the  problems  of  controlling 
an  unknown  linear  dynamical  system. 

The  structure  of  this  chapter  is  as  follows.  In  section  6.2,  we 
clear3y  state  the  problem  under  investigation.  In  section  6.3  we  formu¬ 
late  the  control  problem  and  state  the  solution  .  The  approach  taken  is 
that  of  Open-Loop  Feedback  Optimal  (O.L.F.C.)  control  (see  suction  6.2). 
Using  the  Discrete  Matrix  Minimum  Principle,  we  derive  the  O.L.F.O.  con¬ 
trol  sequence  in  section  6.4.  The  existence  and  uniqueness  of  O.L.F.O. 
control  is  studied  in  detail  ir.  section  6.5,  and  the  asymptotic  conver¬ 
gence  properties  of  the  ovevall  system  in  section  6.6.  Section  6.7  is 
devoted  to  the  discussion  of  approaches  and  of  the  results.  Detailed 
references  are  given  in  section  6.8. 


I 
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Theoretical  results  derived  in  this  chapter  will  be  applied  to  the 
control  of  third  order  linear  systems  with  unknown  gain.  The  computer 
simulation  resalts  and  discussions  will  be  treated  in  tne  next  chapter. 
6.2  Problem  Statement 

Let  us  consider  the  discrete  linear  system 


x(k  +  1)  =  A(k)x(k)  +  b(k)u(k)  +  £(k) 

§:  (6.2.1) 

v(k)  =  C(k)x(k)  +  rs/k) 


where  x(k) ,  J.(k)  £  Rn,  £(k) ,  ^(k)  £  Rn,  A(k)  is  a  known  nxn  matrix,  C(k) 
is  a  known  cum  matrix,  and  u(k)  is  a  scalar  control.  We  assume  that  the 


'gain"  vector  ii(k)  is  unknown,  but  we  know  that  it  satisfies  the  difference 


equation 


b(k  +  1)  =  G(k)b(k)  +  y(k)  (6.2.2) 

where  G(k)  is  a  known  nxn  matrix  and  xOO  -  It  is  assumed  that  the 

vectors  { x(0) ,  b(0),  ±(k) ,  ~(k) ,  v(k);  k  =  0,  1,  ...}  are  independent 
Gaussian  random  vectors  with  known  statistical  laws: 
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Our  objective  is  to  find  a  control  sequence  ;u(0),  . ..,  u(:C  -  1), 
such  that  the  cost 

N-l 

J(u)  =  ~  Ejx’COF  x(X)  (k)W(k)x(k)  +  h(k)u2(k)| 

k~0 

is  minimized  subject  to  (6.2.1)  and  (6.2.2).  The  expectation  is  taken 
over  all  underlying  random  quantities.  We  shall  assume  that  JF,  and  W(k) 
are  nonnegative  definite  symmetric  matrices,  and  that  h(k)  is  a  positive 
scalar  for  each  k. 

Depending  on  the  kinds  of  admissible  controls  that  we  are  allowed  to 
chcose,  different  formulations  of  the  stochastic  optimization  problem  are 
possible.  In  the  most  general  setting,  we  may  assume  that  the  control  is 
a  random  function  of  the  observed  data,  i.e.,  u(k)  =  t(u;F(U(0,k  -  l),k) 
is  a  conditional  probability  measure  on  the  control  space.  If  the  condi¬ 
tional  probability  measure  is  regular,  then  the  control  is  said  to  be  a 
mixed  control  lav;.  If  the  conditional  probability  measure  is  singular 
(Radon  measure),  then  the  control  is  said  to  be  a  pure  control  law.  Un¬ 
fortunately,  little  can  be  done  at  this  level  of  generality  where  we  con¬ 
sider  both  mixed  and  pure  control  laws. 

In  the  next  level  of  generality,  we  may  confine  ourselves  to  consider 
only  pure  control  laws  to  be  admissible,  i.e.,  the  control  at  each  instant 
is  a  fixed  function  of  the  observed  data;  in  this  case,  the  resulting  con¬ 
trol  will  be  a  random  variable  through  its  dependence  on  the  random  ob¬ 
served  data.  This  type  restriction  of  admissible  control  leads  to  Bellman's 
equation  [25}  whose  solution  may  only  be  approximated. 

Finally,  we  may  restrict  ourselves  to  consider  only  deterministic  open 
loop  controls  to  be  admissible;  this  essentially  means  that  we  ignore  the 


-182- 


t 


zero-mean  random  vectors  and  assume  that  the  system  will  behave  according 
to  its  average  behavior.  Of  course,  this  may  not  lead  to  a  good  control 
system,  especially  whenever  the  covariances  of  the  disturbances  are  large. 
To  compensate  for  this,  we  shall  recompute  the  open-loop  optimal  deter¬ 
ministic  control  after  reevaluating  the  state  *  uncertainty  of  the  system 
at  each  and  every  step  (time).  A  control  sequence  which  is  optimal  in 
this  manner  will  be  called  the  open-loop  feedback  optimal  (O.L.F.O.) 
control.  Another  interpretation  of  O.L.F.O.  control  is  the 

following.  Assume  that  we  are  to  control  a  system  without  knowing  whether 
any  further  observations  will  be  available,  or  if  available,  we  do  not 
know  exactly  when  the  data  will  be  observed.  Under  this  situation  the 
principle  of  optimality  is  difficult  to  apply.  One  logical,  and  in  some 
sense  optimal  approach,  is  to  design  an  optimal  control  strategy  based  on 
the  total  information  available  up  to  the  present  time,  and  continue  to 
use  this  strategy  until  new  information  becomes  available  ;  then  we 
change  our  control  strategy  accordingly. 

In  this  chapter,  we  shall  look  for  the  O.L.F.O.  control.  We  shall 
see  that  such  a  control  sequence  is,  in  some  sense,  "adaptive11  in  nature. 

6 . 3  Formulation  of  Control  Problem  and  its  Solution 

The  present  time  is  indexed  by  k.  Let  us  assume  that  the  control 
sequence  U  (0,k  -  1)  i  {u  (0),  u  (1),  — ,  u  (k  -  1)}  has  been  applied 
to  Che  system,  and  that  the  observation  sequence 

Yl.*(0  ^^(O.k)  -  '-%*(()  i-l)^'i=0  obsferved*  We  would  like  to  find  a 
"future"  control  sequence  U  (k,N  -  1)  k  (u(k),  — ,  u(N  -  1)}  so  as  to 
minimize  the  future  cost  (cost  to  go)  conditioned  on  the  total  available 
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information  at  the  present  time.  Let  us  denote  the  "-algebra  generated 
by  the  observed  data  ^^(O.k)  as  FOc.lAo.k  -  1));  the  symbol 

U  (0,k  -  1)  is  used  to  denote  that  the  data  is  really  dependent  on  the 
past  control  history.  Our  aim  now  is  to  find  the  contrcl  sequence 
l.'(k>N  -  1)  such  that  the  cost  to  go 


is  minimized  subject  to  the  constraints  (6.2.1)  and  (6.2.2).  The  cost  has 
the  simple  form  (6.3.1)  because  the  future  control  sequence  U(k,N  -  1)  Is 
assumed  to  be  deterministic.  (If  the  future  controls  were  assumed  to  de¬ 
pend  observed  data,  we  could  not  take  the  last  term  of  (6.3-1)  outside 
the  expectation  operation.)  It  is  now  possible  to  formulate  the  pioblem 
so  that  deterministic  optimization  techniques  can  be  applied. 


Let  us  define  for  j  £  k, 

x(jlk,u"(0,k-l))  A  E{x(j) |F(k,U*(0,k-l)}  (6.3.2) 

b(j |k,lf  (0,k-l))  k  E{b(j) |F(k,U*(0,k-l))}  (6.3.3) 

£j;(j  |k,u''(0,k-l))  k  x(j|k,U*(0,k-l))  -  x(j)  (6.3.4) 

®^(j |k,U(0,k-l))  k  £(j|k,lf(0,k-l))  -  b(j)  (6.3.5) 


We  note  that  S(j|k,U  (0,k  -  1))  is  F(k,U  (0,k  -  Immeasurable  if  j  >  k, 
so  for  j  a  k,  we  have 
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Elx'(j)M  x(j)|F(k,U*(0,k-l))}  -  x{j|k,U*(0,k-l))M  x(j |k,U*(0,k-l))  + 

E{e^(.i|k,U*(0,k-i))M  e^(j  |k,U*(0,k-l))  | F(k,UX(0,k-i))  }  (6.3.6) 

where  M  is  an  arbitrary  nxn  matrix.  If  we  define  the  state-error  second- 
moment  matrix 

Z  (j jk,U*(0,k-l))  4  E{e  ,(j |k,U*(0,k-l))e' (j |k,U*(0,k-l)) ] F(k,U*(0,k-l) ) } 

—  X  "X 

(6.3.7) 

then  using  (6.3.6)  and  (6.3.7),  the  conditional  cost  (6.3.1)  can  be  written 
as  follows 

J(U(k,N-l);U*(0,k-l),k)  =  x' (Nik,U*(0,k-l))F  x(h’!k,U*(0,k-l)) 

N-l 

\  tr  F  !^(S|k,U*(0,k-l))  +  7^  {x’(j|k,U*(0,k-l))W(j)x(j|k,U*(0,k-l))  + 

3=k 

tr  _W(j  !  k  ,U  (0 ,  k-1)  )  +  h(j)u2(j)J  (6.3.8) 

To  complete  the  formulation,  we  shall  have  to  derive  dynamical  equations 
satisfied  by  5<(j  |k,U  (0,k-l))  andI^(j|k,U  (0,k-l)), 

Since  all  the  noise  sequences  are  assumed  to  be  uncorrelated  and 
white,  we  have  (see  chapter  2,  section  2.3) 

E{i.(j)!F(k,U*(0,k-l))}  =  0  ;  E{x(j)  iF(k,U*'0,k-l)) }  -  0  ,  jik 

(6.3.9) 

The  admissible  controx  is  assumed  to  be  deterministic;  hence,  (6.2.1), 
(6.2.3)  and  (6.3.9)  imply  that,  for  j  £  k, 

x(j+llk,U  (0,k-l))  =  A(j)x(j|k,U  (0,k-l))  +b(jik,U  (0,k-l))u(j) 


(6.3.10) 
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b(j  +  1 |k,u"(0,k  -  1))  =  G(j)b(jik,u'"’(0,k  -  1))  (6.3.11) 


with  initial  condition  (at  the  present  time  j  =  k) 


x(kjk,U  (0,k-l))  =  E{x(k)|F(k,U  (0,k-l))}  ; 

b(k(k,U*(0,k-l))  =  E{b(k)  | F(k,U*(0,k-l) ) } 


(6.3.12) 


From  equations  (6.3.2)  to  (6.3.5),  (6.2.1),  (6.2.2),  (6.3.10),  and  (6.3.11), 
we  obtain  the  difference  equation  for  the  error  vectors  for  j  '  k: 


e^j+llk.U  (0,k-l))1  j-A(j)  :  uQjnre^alk.U  (0,k-l) ) 


eb(j+llk,U  (0,k-l))J  [_  0  I  G(j)  j|_eb(j  k,U  (0,k-l)). 


"I(j)‘ 

...  (6.3.: 

-v(j). 


The  initial  error  at  j  =  k  only  depends  on  {_£(!) ,  y(i)},  i  5  k  -  1,  and 
tjl(i)},  i  <  k,  and  so  it  is  independent  of  {_£ (i) ,  _y(i)},  j  >  k.  Also, 
since  all  noises  are  uncorrelated,  zero  mean,  and  white  Gaussian,  (6.2.6) 
and  (6.2.7)  imply  that 


E{l(j)l'(j)|F(k,U:(0,k-l))}  =  R(j) 


E(x(j)jl'(j)|F(k,U5'(0,k-l))}  =  N(j) 


If  we  define  the  second-order  moment  matrix  (for  j  i  k) 


(6.3.14) 


i(j|k,U  '0,k-l))  -  E 


“e  (j|k,U  (0,k-l) )' 


^(jlk.o'^o.k-i)). 


(e^(j|k,U  (0,k-l)):e^(j!k,UX(0,k-l))] 


I  F (k , U  (0,k-l)) 


(6.3.15) 


S 


» 
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Equations  (6.3.13),  (6.3-14)  and  the  independence  of  initial  error 
(j  =  k)  and  of  the  future  noise  sequence  iirply  that  E_(j|k,U  (0,k-l)), 
j  t  k,  is  generated  by  (see  chapter  2,  section  2.3) 

f(j+lik,U*(0,k-l))  =  A(j,u(j))E(j|k,U*(0,k-l))A,(j,u(j))  +R(j) 

•  (6.3.16) 

where 

A(j)  :  u(j).ci  rm  •  o 

A(j,u(j))  4  ...  ...  ;  R(j)  4  ...  _  ...  .  (6.3.17) 

-0  G(j)J  [o  :  N(j) 

The  initial  condition  is  given  by 

(k|k,U*(0,k-l)) |e^(k|k,U*(0,k-l)) ] 

| F(k,U*(0,k-l))  (6.3.18) 

From  (6.3.12)  and  (6.3.18),  we  see  that  x,(k|k,U  (0,k-l))  and 
,  * 

bi(kjk,U  (0,k-l))  are  the  conditional  means  of  jc (k)  and  _b (k)  ,  respectively, 
while  £(kjk,<J  (0,k-l))  is  the  conditional  covariance  matrix  of  the  aug¬ 
mented  vector 

(k)' 

(k)_ 

These  quantities  can  be  generated  by  the  following  identification  equations, 
(6.3.19)— (6. 3.23) ,  once  the  past  control  U  (0,k-l)  has  been  chosen: 


r®x(k|k,U*(0,k-l))" 

I(klk,U*(0,k-l))  =  E  ...  [e^ 

(k !  k ,  U*  (0 ,  k-1) )_ 
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’x(i+l  ji+l,U  (0,i))' 


b(i+l|i+.’.,U  (0,i)i 


=  [I,  --V  (i+l|l,u  (0,i))C(i+l)]A(i,u*(i)). 


rx(i|i,U*(0,i-l))‘ 

...  +V  (i+l|i,U  (0,i))^(i+l)  ;  i  =  0,  1,  . ..,  k  -  1 

Lb(i!i,U*(0,i-l)l 


x(o|0,UW(0,-l)1  fx(0  j  0) 

b(0  0,U*(0,-1)J  Ife(OjO). 


fx  • -Z  C'  (0)  (C(O)E  C'  (0)+g(0))"X(C(0)x  -x(0)) 


(6.3.19) 

where 


C(i  +  1)  &  (C(i  -!  1)  I  0  ]  ;  i  =  0,  1,  ....  k  -  1  (6.3.20) 

*  mn 

"k  k 

and  V  (i+lji,U  (0,i)),  i  -  0,  1,  ...,  k  -  1,  is  a  solution  of  the  following 
equations; 

v'"(i+l|i,u't(0,i))(C(i+l)A(i!i,U*(0>i))C’(i+l)+g(i+l))  =  A(i|i,u’C(0,i))C'  (i+1) 


;  i  =  0,  1,  . . . ,  k  -  1 


(6.3.21) 


A(i|i,0  (0,i))  =  A(i,u'  (i) )_E(i  |  i,U*(0,i-l))A'  (i,u*(i))  +  R(i) 


i  =  0,  1 . k  -  1 


(6.3.22) 


£(i+i|i+l,U  (0,i))  =  A(i|i,U!!f(0>i))-V*(i+l|i,u"(0,i))C(i+l)A(i|i>U*(0,i)) 


i  -  0,  1 . k  -  1 


(6.3.23) 
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Z.(0|0,u‘(0,-1  £  £(ojO)  = 


-1, 


I  -2  C' (0) (C(O)L  C'(0)+Q(0))  C(0)Z 

- XO  - XO—  —  — _ 


:  0 
XO  .  — 


ho- 


Referring  to  chapter  3,  section  3.3  we  note  that  the  identification 
equations  represent  an  optimum  observer-estimator  for  the  augmented 
system: 


'  x(i+l)~ 

f  x(i)' 

*£(i)‘ 

_b(i+l)_ 

=  A(i,u  (i)) 

1 

ler  | 

/-v 

W* 

1 _ 

+ 

_l(i). 

(6.3.24) 


y(i)  =  C(i) 


f*x(i)' 

b(i). 


+  n(i) 


If  either  (£(i)  >  Q,  i  =  0,  1,  . ..,  k  -  1,  or  C(i+l)R(i)C r (i+1)  >  0, 
i  =  0,  1,  ...,  k  -  1  (or  both),  the  unique  V  (i+i ! i , U  (0,i))  which 
satisfies  (6.3.21)  to  (6.3.23)  is  given  by 

V*(i+l|i,U*(0,i))  =  A(i|i,UX(0,i))C' (i+l)« 

[C(i+l)A,(i  |  i,U*(0,k-l)C'  (i+l)+£(i+l)  ]_1 


i  »  0,  1,  ...,  k  -  1  (6.3.25) 


and  the  identification  equations  specify  a  Kalman  filter  for  the  aug¬ 
mented  system  g. 

In  all  cases,  where  the  driving  and/or  observation  noises  may  be 

J. 

degenerate,  the  conditional  covariance,  I_(kjk,li  (0,k-l)),  given  by  (6.3.21)- 
(6.3.23)  is  unique;  and  the  conditional  mean 
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■x(k|k,U*(0,k-l)' 

_b(k|k,U*(0,k-l)_ 

is  unique  almost  surely. (see  chapter  3).  Thus,  we  may  assume  that  these 

•k 

quantities  are  known  if  U  (0,k  -  1)  has  been  chosen.  We  can  then  formu¬ 
late  the  following  deterministic  control  problem  at  the  kth-step: 

Open-Loop  Control  Problem  (k  -  j  ^  N  -  1) ; 

Given:  x(j+ljk)  =  A(j)x(j |k>  +  b(j |k)u(j)  (6.3.26) 

b(j+l|k)  =  G(j)b(j|k)  (6.3.27) 

£(j+l|k)  =  A(j ,u(j))£(j |k)A' (j,u(j))  +  R(j )  (6.3.28) 

with  known  initial  conditions  at  j  =  k 

x(k!k)  =  X(k|k,u*(0,k-1))  ;  £(k jk)  =  b(kjk,u*(0,k-l))  ; 

Z_(k|k)  =  £(k|k,U*(0,k-l))  .  (6.3.29) 

We  are  to  find  a  deterministic  control  sequence  U(k,N  -  1)  such  that  it 
minimizes 

J(U(k,N-l)  ;  U*(0,k-1)  ,k)  =  1 1  S'  (N |k)F  S(N|k)  +  tr{Fj,(Nik)|  + 

N-] 

(*'  (j  |k)W(j)S(j  jk)  +  tr|W(jU(j  jk)|  +  n(j)u2(j)^  (6.3.30) 

j=k 

subject  to  the  constraints  (6.3.26)  to  (6.3.28),  where  the  matrices  £  and 
W( j )  are  defined  by 

Wc  shall  not  explicitly  stress  the  dependence  on  the  past  control  history 
U  (0,k-l);  for  this  reason  the  symbol  U*(0,k-1)  shall  be  dropped  without 
causing  any  confusion. 
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-1 

;  K' 

oj 


Etj)  = 


tf(j)  o'] 

.  o 


(6.3.31) 


For  rise  above  deterministic  control  probiea,  we  shall  denote  Its  optical  ccn- 
trol  sequence  by  l’°(S;,S-l>  s  iu°(]  k)  :^r,  where  the  superscript  o  Is  used  to 

jl“£  _ 


o 

denote  cntiml  for  the  open-icon  control  problem:  the  symbol  g  (i,',k)  is  wsed 
to  indicate  that  the  control  is  eoes-icgo  optimal  conditioned  on  She  ohse : 
vation  "p  to  the  present  time  k. 

The  solution  for  the-  above  deterministic  optiaai  control  problem  is 
given  be  lew;  the  detailed  derivation,  will  be  carried  oat  in  section  6.4. 

Ice  optimal  control  sequence,  L'  Ck.S-l),  is  given  by 


u°(j  k)  =  -Tnij  k)-fb°5  (j  »k)£(jwi:k)b°(j  k> ]~l 


-x°a  k)1 


i°*  (j  :k)K(j-rl!k)  ©  (j  Ik}| 


-h(j  k)d'  (j-H.)}  ... 


Ls°(j  MJ 

(6.3.32) 

where  K(jik),  j  =  k  +  1,  . ..,  X  -  1,  satisfies  ciie  natrix  difference 
equation 


K(j  k)  =  @  ,(j:k)iK(j+l|k)-K(j+lik)b0(j(k){h(i!k)+b0,(jik)k(j+l|k)b°(jIk)} 


fc°’(j  k)K{j-rlIk)]<2)(jlk)+D(jik)  ;  K(N|k)  = 


F  0 
0  0 


0  0 


(6.3.33) 


and  for  j  =  k,  ....  N  -  1 
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®  (5 fk>  =  A(j)~£°(j  ik)h  1(i  ik)d'  (jr!)  ; 


A(i)  0 

0  A'j)g 
A(j)  *  ...  - 

0  £(j)g 


A0>8„, 


lo  — 


BCjlW  -  i(j)  -  d(j+i)ii  1{j|K)d-(ia) 


(6.3.34) 


(6.3.35) 


0 

A(j-DSa)e. 

A(j-l)S(j)e^ 


R  ;  o  (31k) 


b°(j|k) 
if  (j  'k)G'  (j)^ 


I  if  (j  (j)e 


_  ?n(nAi) 


£°0|k) 


(6.3.36) 


h(j|k)  *  h(j)  +  tr{Ef(j  jk)S(j+l)} 


(6.3.37) 


The  natrices  lf(i{k),  S_(j  +  1),  j  =  k,  k  +  1,  ...,  N  -  1,  are  given  by 


lf(j+l|k)  =  G(j)lf(jJk)G*  (j)+>:(j)  j  *  k,  ....  N  -  1  ; 


i£(k|k)  =  T^(k|k,U  (0,k-l)) 


(6.3.38; 


S(j)  =  A'(j)S(j+l)A(j)  +  W(j)  i  =  k,  ...,  N  -  1  ;  S(N)  =  F 


(6.3.39) 


and  the  vector  £  (j|k)  satisfies 


'The  vectors  e, ,  e_,  ....  e  represent  the  natural  basis  in  R  . 

—1—2  — n  n 


where  u°(k  k)  is  given  by  (6-3.32)  to  (6.3.40).  The  structure  of  the 
O-L.F-O.  control  system  is  described  by  Figure  6.1.  Though  the  equations 
are  complicated,  the  digital  computer  implementation  of  O.L  F.O.  control 
sequence  is  actJjaily  straightforward.  A  flow  chart  description  of  the 
O.L. F.O.  control  is  given  in  figures  6.2  and  6.3.  In  the  if  Hewing,  we 
shall  outline  the  computational  procedure  to  find  the  O.L. F.O.  control 
sequence . 

A 

1.  If  k  =  0,  y(0)  is  observed,  and  x(0  G,U  (0,-1), 

b(0]0,U  (0,-1),  1(0 jO, U  (0,-3)  are  given  by  (6.3.19) 
and  (6-3.23).  If  k  >  0,  assume  that  U  (0,k-l)  is 
chosen  and  *s  observed;  compute 

x(kik,U  (0,k-l),  £(k|k,U  (0,k-l)>  and 

£(kjk,U  (0,k-l))  using  the  identification  equations 
(6.3.19)  to  (6.3.23). 

2.  Compute  ©Cjlk),  D(jjk).  b°(j|k),  h(j|k)  for 
j=k,  k+1,  . ..,  ft  -  1  using  equations  (6.3.34) 
to  (6.3.40). 

3.  Compute  K(k+llk)  using  (6.3.33),  and  the  O.L. F.O. 
control  to  be  applied  at  step  k  is  given  by  (6.3.41). 
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Fig.  6.2  FLOV  CHAPT  FOP.  ON-LINE  COMPLIATIO:  CF  O.L.F.O.  CONTROL 


B‘(K-jjk)K0i5CN-jJk))’l&,tN-jJk)K0)J  (5)  (N-j{k)  ♦  ff{N-jjk) 


K(N  -  k  ♦  1) 

(“)  tkJk’ 

i  \  r 


icbtMk) 

_L_ 


i(k). -(h{k|k)  K'Oc^JJKtN-k+DfCklkjr'f’tklkJKtN.k+l)  0  (kjk) 


jrtkjk)  = 


SxhfWk)^ 


5xb«^)- 


uc(k{k>r-  {(S’(kJk)  +  ff'(kik)K  {N-k+l^tkjkjr^’tkjklKCN-k+l)  0  (k|k» 
TT{k)k)  d'  (k+1)}  [2.  1 

UH 


*(k) 


uc(kjk) 


Fig.  6.3  FLOW  CHART  FOR  COMPUTING  THE  O.L.F.O. 
ADAPTIVE  GAIN  AND  CORRECTION  TERM 
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error  will  be  taken  into  account  in  the  deterministic  control  problem. 


Thus,  this  does  not  correspond  to  pure  separation  as  it  is  in  the  case  of 
stochastic  control  of  linear  system  with  known  dynanics(chapter  5). 

Let  us  first  look  into  the  identification  equations  (6.3.19)-(6.3.23) . 
Suppose  that  ^>(k)  >  ().  If  u(i)  =  0,  then  from  (6.3.21),  we  have 


\f(i+l|i,U(0,i))  = 


■V*(i+l|i,U(0,i))' 


(6.3.42) 


and  so  (5.3.19)  implies 


b(i+l|i+l,U(0,i))  =  G(i)b(i|i,U(0,i-l)) 


(6.3.43) 


Therefore,  a  nonzero  input  is  necessary  to  identify  the  gain  parameter 
vector  b(k) .  From  the  equation  (6.2.1),  we  see  that  if  u(i)  is  very  large, 
then  for  the  most  part  the  value  of  x(i  +  1)  will  be  due  to  lj(k)u(k), 
and  so  the  observation  y(k)  will  contain  a  large  amount  of  information 


about  the  gain  parameter  Mk) .  Therefore,  we  would  expect  that  large  input 
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magnitudes  will  be  helpful  for  the  identification  of  b(k) .  For  a  control 
sequence  {u(k)}*_Q,  if  its  total  energy  is  high,  we  would  expect  such 
control  sequence  to  be  useful  for  identification  purpose.  Buc  large  con¬ 
trol  energy  will  also  give  rise  to  a  high  cost  (6.3.1).  From  the  control 
point  of  view,  we  would  like  to  use  just  enough  control  energy  to  regu¬ 
late  the  state  of  the  system.  In  general,  there  is  a  conflict  between 
identification  and  control,  and  a  reasonable  control  sequence  should 
appropriately  distribute  its  total  energy  to  identify  and/or  control  of 
the  system  S- 

Let  us  consider  (6.3.32)-(6.3.33) .  Comparing  with  the  Levis' 
results,  we  note  that  u°(jjk)  is  the  optimal  control  for  the  problem  of 
controlling  the  system  S^: 


Sfc:  x(j+l|k)  =  A(j)x(jjk)  +  b(j  |k)u(ji  k) 


x(j+ljk)  £ 


*°(j+l|k)' 

•  •  • 

j_£°  (j+l!k)J 

(6.3.44) 


with  the  cost  criteria 

N-l 

J  =  x’(N|k)F  x(N|k)-^{x(i|k)W(ilk)k(iik)+h(j|k)u2(j  jk)+2x’ (i|k)d(i+l)u(i jk) } 

1=k  (6.3.45) 

Therefore  we  can  visualize  h(j|k)  as  the  modified  relative  weighting  on 
the  control.  From  (6.3.31),  we  note  that  h(j|k)  relates  in  a  direct 
manner  with  J^(j|k).  In  a  statistical  sense,  E^(k|k)  reflects  the  level 
of  confidence  we  have  about  the  estimate  of  ]>(k) .  The  modification  on  the 
relative  weighting  on  the  control  is  such  that  heavy  weightine  is  out  in 
the  control  if  we  have  little  confidence  on  the  estimate  of  b (k) ;  therefore, 
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the  control  action  will  be  very  cautious  and  control  energy  will  not  be 
used  unless  it  is  very  necessary. 

Let  us  write 

u"  (k  i  k)  =  -  (h  (k !  k)+b° 1  (k  |  k)  K  (k+1 !  k)  b°  (k  |  k)  )  _1  *b° '  (k  |  k) K(k+1  |  k)  @  (kjk) 


C*lfx°(k!k 


0.|Lo°(k!k 


-  (h (k | k)+b°  * (k 1 k)K(k+l | k)b° ' (k | k) )  (k+1 1 k) 


0  :  0 


0  :  I 

—  .  n 


©  (k  |  k) . +h~1  (k  |  k)  d 1  (k+1) 


*°<kikj 


o  (k|k). 


(6.3.46) 


We  shall  call  the  row  vector  (lxn) 


o(k)  |  -(h(k|k)+b0,(k|k)K(k+lik)b0(kjk))_1-b0,(k|k)K(k+l|k)  ©(kjk) 


_I 

— n 

)  ... 
L  P_. 


the  O.L.F.O.  adaptive  gain,  and  the  term 


(6.3.47) 


uc(k;k)  =  -{ (htk|k)+b°' (k | k)K (k+1 | k) b° (k j k) }  1-b°' (kjk)K(k+ljk) ©  (k|k) 


'0  :  o']  rs°(k|k)' 

.  +h(k|k)d' (k+1) 

0  :  I  o°Odk) 

—  — n J  —  '  ■ 


(6.3.48) 


the  correction  term.  Thus,  the  O.L.F.O.  control,  (6.3.46),  becomes 


u  (k [ k)  =  £(klk)x°(k|k)  +  uc(k|k) 


(6.3.49) 


From  (6.3.33)-(6.3.37) ,  we  note  that  Z^(kjk,U  (0,k-l))  affects  indirectly 


the  O.L.F.O.  adaptive  gain  and  the  correction  term.  The  cross-error 
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covariance,  I^kCkjk.i’  (0,k-l)),  only  affects  the  correction  tern;  and  if 
E^^klk.U  (0,k-l))  is  zero,  then  from  (6.3.36)  and  (6.3.48)  we  conclude 
uC(k|k)  =  0. 

■  * 

Assume  that  L  (i lk,U  (0,k-l))  =  0,  then  from  (6.3.33)-(6.3.37) ,  we 
b  — 

have  inductively 


Ht(k)  :  o' 


K(k|k)  = 


(6.3.50) 


where  K(k)  is  given  by  (5.3.6),  and  from  (6.3.47),  the  G.L.F.O.  adaptive 
gain  is 

c(k)  =  -(h(k)  +  b'(k)K(k  +  l)b(k))'V(k)K(k  +  DAfk)  (6.3.51) 


which  is  the  truly  optimum  gain  (see  chapter  5,  section  5.3).  The  assump- 
tion  that  T^(kjk,U  (0,k-l))  =  0  also  implies  I^^(k|k,U  (0,k-l))  =  JO,  and 
so  the  correction  teirm  is  zero,  and 

u*(kjk)  =  -(h(k)  +  b'(k)K(k+l)b(k))-1b'(k)K(k+l)A(k)i0(k|k)  .  (6.3.52) 


Thus  we  see  that  if  for  some  k,  the  identification  of  j)(k)  is  assured  to 
be  exact,  i.e.,  the  level  of  confidence  on  the  estimated  gain  parameters 
is  very  very  high,  then  the  O.L.F.O.  control  will  act  optimally  and  use 
the  obtained  estimate  of  b_(k)  as  if  it  were  the  true  gain  vector. 

Finally,  we  would  like  to  comment  on  the  computational  requirements 
of  the  proposed  scheme.  The  computation  of  the  O.L.F.u.  control  is  done 
on-line.  At  each  time  unit  k,  we  have  to  solve  a  one  step  2n-vector  dif¬ 
ference  equation  and  a  one  step  2n  *  2n  matrix  difference  equation, 
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(6.i.i9}-{6.1-23)',  caoutir?  the  parade. ters,  (6.3.3?)-(6.3.40),  which  in¬ 
volve  seme  oae-  seep  computation,  (6.3.34)-(6.3.37),  and  an  X  -  k  steps 
c-vecEcr  difference  equation  (6-3-40)  and  an  21  -  k  steps  nxn  matrix  dif¬ 
ference  equation  (6.3.33);  finally  we  have  to  solve  an  K  -  k  step 
(a  -5-  i)a  *  (a  t  l)n  matrix  difference  equation  (6-3.33).  (N’ote  that  the 
natrix  difference  equation  (6-3.35)  can  be  precomputed  off-line.)  Tha 
O-L-F-O-  control  is  then  computed  using  (6.3.32).  The  total  storage 
capacity  needed  corresponds  to  the  storage  of  the  state  end  parameter 
estimates  (2a)  and  the  error  covariance  matrix  (2n  *  2n).  The  capability 
of  computing  the  O-L-F-O.  control  sequence  in  almost  real  tine  will  de¬ 
pend  on  the  complexity  of  the  system  beiGg  considered  and  the  computation 
speed  of  the  digital  computer  used  to  implement  the  O.L.F.O.  control  (see 
also  chapter  7). 

6-4  (hea-iooa  Optimal  Control 

In  this  section,  we  shall  derive  the  open-loop  optical  control  for 
the  deterministic  control  problem  (6-3-26)-(6. 3.31) .  The  deterministic 
formulation  allows  us  to  use  the  discrete  matrix  minimum  principle 
(sheorem  2.4.1)  to  derive  the  set  of  necessary  conditions  for  optimality. 

Let  us  form  the  Hamiltonian  for  the  deterministic  control  problem 
(6.3.26)-(6.3-31)  for  j  -  k,  k  -r  1,  ....  8  -  1. 

H.  =  < By. ( k) , A( j )  x(  j  j k)+b ( j  j k) u ( j ) -x( j  j k^-f^ ( j+1 1 k)  , G ( j ) b ( j  i k) - 

b(j  k)>+tr  j(A(j  ;u(j))Z(j  ik)A'  (j  ,u(j))+R{j)-Z(j  |k))P'  (j+l|k)j 
+  f<lU:k),W(j)*(j!k)>  +  |  h(j)u2(j)  |  tr  £(j)Z(j|k)  (6.4.1) 


>1- 

jk)  is  the  costate  vector  associated  with  x(j|k),  £^(j'k)  is 
the  costste  vector  associated  with  £(j|k)  and  P(j|k)  is  the  costate 
matrix  associated  with  2!(j|k).  Use  of  the  discrete  matrix  minimum  prin¬ 


ciple  leads  to  the  following  relations 
(a)  The  canonical  equations  are: 

x°(j+l|k)  =  A(j)x°(j|k)  +  £°(j|k)u0(j|k)  (6.4.2) 

b°(j+l|k)  -  G(j)bC(j|k)  (6.4.3) 

i°(j+l|‘k)  =  A(j,u°(j|k))E°(j|k)A,(j,u0(j!k))  +R(j)  (6.4.4) 

£.°(j|k)  -  A’(j)j^(j+l|k)  +  W(j)x°(j|k)  (6.4.5) 

*  ^(j)^  (j+1  Ik)  +£^(j+l|k)u°(j|k)  (6.4.6) 


P°(j|k)  =  A,(j,u°(j|k))P0(.i+l|k)A(j,u°(j|k))  +|w(j)  (6.4.7) 

(b)  The  boundary  conditions  are: 


where  g.  (j 


at  time  k:  *°(k|k)  »  x(k|k,U*(0,k-l)) ;b°(k|k)  =  b(k|k,U*(0,k-l)) 

JL°(k|k)  =  £(k|k,U*(0,k-l))  (6.4.8) 

ac  time  Hi  ^(sjk)  -  F  *° (N | k)  ;e£ (N | k)  =0  ; 

P°(N|k)=|F  (6.4.9) 


(c)  In  minimizing  the  Hamiltonian,  we  set  (for  j  i  k) 


-  6°,(j|k)^(j+l|k)+2tr{pJ1(j+l|k)A(j)Z^b(j|k)+ 

u°(j  jk) 

£( j ( j I k) P°  ( j+1 1 k) }+u° ( j | k) • { h ( j )+t r [ 21° ( j | k) P°  ( j+1 i k) ] }  =  0 


(6.4.10) 
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where  we  have  decomposed  the  costate  matrix  as 
follows 


P°  ( j  !  k) 


~fn(3  !k)  :  p°2ci  ik>- 

:  l22(j|k)- 


(6. A. 11) 


From  (6.4.7)  and  (6.4.9),  we  deduced  that  P°(jjk)  is  nonnegative  definite 
since  F  and  W(j)  are  assumed  to  be  nonnegative  definite.  Therefore, 

=  h(j)  +  2  tr{l£(j | k) ( j  +  l|k) }  >  0  (6.4.12) 

and  so  the  control  u°(j|k)  given  by 


3“H 


?u“(j) 


u°(j|k) 


u°(jlk)  =  -(h(j)+2tr(Z^(j|k)P°1(j+l|k))]  1{b0,(j|k)£^(j+l|k)+ 

2 1 r (?°i  ( j+1 1  k)  A( j )£ ( j  ! k)+G(j )  ( j  !  k) P°2  ( j+1  j  k) ) }  ; 

j  >  k  (6.4.13) 


indeed  minimizes  the  Hamiltonian. 


From  (6.4.4)  we  obtain  equations  for  ^^(jjk)  and  ^(jjk)  for  j  1  k: 


Cu  ( j+1  i  k)  =  A( j  )l".  ( j  I k)G '  ( j )  +  U°  ( j  I  k)  ( j  i  k)G  ’  ( j  )  (6.4.14) 


if  (j+1 1  k)  =  G(j)zf(j|k)G'(j)  +N(j) 


(6.4.15) 


with  initial  conditions 


z£b(k|k)  =  j^K(k|k,U*(0,k-l))  ;  Z^(k|k)  =  J^(k|k,U*(0,k-l)) 


— xb 


■=b 


(6.4.16) 
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From  equations  (6.4.7)  and  (6.4.9),  we  obtain  the  equations  for 
P^j  |k)  and  for  ^  =  k*  k  +  1*  S: 

pJjCjJk)  -  A'CjjP^y+llWACi)  +  |w(j)  (6.4.17) 

P°2(j|k)  -  A‘  (j)P°2(j+l|k)G(j)  +  u° (j  |k)A*  (j^tj+lik)  (6.4.18) 
P°x(N|k)  =  ^F  ;  P°,(N|k)  =  0  .  (6.4.19) 

We  note  from  (6.4.13)  to  (6.4.19),  that  the  values  of  £^(jlk)  and 
P.22(j  lk)  are  irrelevant  in  computing  the  open-loop  optimal  control  se¬ 
quence  u°(j|k).  From  (6.4.17),  P.°^(j|k)  is  independent  of  the  observa¬ 
tion  and  the  control,  and  thus  it  can  be  precomputed.  To  emphasize  this 
fact,  we  define 

S(j)  (k  2Pj1(j|k)  >0  ;  N  >  j  >  0  (6.4.20) 

and  so  S(j)  satisfies  the  matrix  difference  equation 


S(j)  =  A'(j)S(j  +  l)A(j)  +W(j)  ;  S(N)  *  F  .  (6.4.21) 

From  (6.4.15),  jf^(jjk)  only  depends  on  the  observation;  thus  it  is 
meaningful  to  define  a  "modified  control  weighting": 

h ( j  | k)  k  h(j)  +  tr(l£(j|k)S(j  +  1))  >  0  .  (6.4.22) 

Let  us  define 


l°(j  |k) 


’2^12  ( j  I  k) 

-2— 12  ^  I  k)-^n- 


(6.4.23) 


Then  by  using  (6.3.34)  to  (6.3.36),  (6.4.2),  (6.3.13)  to  (6.4.23),  we  obtain 
the  set  of  matrix  difference  equations:  j  =  k,  k  +  1,  . ..,  N  -  1. 


-2G4- 


~ x" (j+1 ! k/ 

m  •  • 

=  ©(j!k) 

k°  (j  !k’)~ 

•  »  ♦ 

-  b°  (j  |k)h_1(j  jk)b°*  (j  |k) 

‘^(j+J-jk) 

•  *  • 

_£°  (j+ljk). 

_o°(i!k)_ 

_b°(j+l|k). 

\E^(j  i  k J 

~(S>  *  ( j  !  k) 

(j+ljk)' 

•  »  • 

+  D(j|k) 

~x°  (j  |k)’ 

»  •  • 

.1°  (jin 

,o°(j+lik)_ 

_o°  (j  |k)_ 

(6. A. 24) 

(6.4.25) 


with  boundary  conditions  at  time  M: 


r-  O 


l°(NjkiJ 


I  o 

0  0 

•  •  •  •  « 
0  0 


’x°(N|k)- 
•  •  • 

_2°(N*|k)_ 


(6.4.26) 


From  (6. 4. ’4)  to  (6.4.26)  and  (6.4.3),  (6.4.16),  we  can  solve  for 
£^(j|k),  0,  (jjk),  x°(jjk)  and  o°(j+l jk) .  To 'bypass"  the  two  point  boundary 
valua  problem,  we  define  the  matrix  K(j|k)  by 


=  K(j|k)| 


_£°  ( j  I  k). 

Substituting  (6.4.27)  into  (6.4.24)  and  (6.4.25)  we  obtain 


L2°(j |k) 


(6.4.27) 


(j+1  In¬ 


ti  +  £°  ( j  I  k) h  1  ( j  |  k)  b°  ’  ( j  |  k)K  ( j+1 1  k)  ]] 


^(ilk)- 


=  ®(j|k) 


U°  (j+l|k)J 

-  1 

*k°  ( j  j  k)' 

‘S°(j+ljk7 

(K(j|k)  -  D(j|k)] 

•  •  • 

=  ©’(j|k)K(j+l|k) 

•  •  « 

£  ( j  1  k)_ 

.0°  (j+1  (k). 

(6.4.28) 


.  (6.4.29) 
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If  [I_  +  b°(j|k)h  *(j  |k)b°*  (j  |k)K(j+l|k)  j  has  an  inverse,  then  (6.4.28), 
(6.4.29)  imply  that 


{£( j I k) -D ( j | k)- © ’ ( j 1 k)K( j+1 1 k) [I+b° ( j | k) h-1 ( j | k)b°  * ( j ! k)K( j+1 ! k) ] ' "1 ©  ( j ! k) } 


,-l. 


(6.4.30) 


Since  x° (j  jk) ,  c° (j |k)  can  be  arbitrary,  (6.4.30)  implies  that  the  matrix 
difference  equation  holds: 


K( j | k)  =  ©  '  ( j  i k) (K( j+1 1 k) -K ( j+1 1 k)b° ( j | k) {h ( j | k)+b° ' ( j | k)K ( j+1 ! k)b° ( j | k) > 


,-l 


£0,(j!k)K(j+l(k))(g)  (j|k)+B(j|k)  ;  K(N|k) 


where  we  have  used  the  matrix  identity: 


( I  +  A  B')"1  =  I  -  A(I  +  B'A)~1B’ 
~n — n r — 


F  0  ...  0* 

0  0  ...  0 

•  •  •••  •••  ••• 

0  0  ...  0 

(6.4.31) 


A,B  s  M  .  (6.4.32) 

—  —  nr 


The  identity  (6.4.32)  is  true  provided  one  of  the  inverses  exists.  The 
two  point  boundary  problem  is  now  transformed  to  the  problem  of  finding  a 
solution  of  the  matrix  difference  equation  (6.4.31).  The  existence  and 
uniqueness  of  (6.4.24)  to  (6.4.26)  and  (6.4.8),  (6.4.16)  can  be  deduced 
from  the  existence  and  uniqueness  of  K(j|k),  N  >  j  >  k,  satisfying  (6.4.31). 

The  optimal  open-loop  control  is  given  by:  (N  -  1  >  j  >  k) 
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u°(j|k)  = 


p^C3+1ik>] 

X(j  |k)  _b°  *  (j  |k)  —  +^’ (j+1)  ... 

_o°(j+lj.kl  .2.°  ( j  I k 


=  -h-1(j  lk)b°'  (j  jk)K(j+l|k)  [I+b°(j  |k)h_1  (j  |k)b°’  (j  |k)K(j+l|k)  ]  X- 


*S°  (j  ik)" 

®(j!k)  ...  -h 

c°  ( j  |  k). 


’*°  (j|kT 

_1(j  |k)d/  (j+1)  ... 

o°  (jjk). 


(6.4.33) 


Using  (6.4.32)  and  also  the  matrix  identity1 


I  -  A(A  +  B)-1  =  3(B  +  A)-1 


(6.4.34) 


(6.4.33)  becomes  (21  -  1  t  j  t  k) 


u° ( j  Ik)  =  - (h ( j  j k)+b° *  ( j  1  k)K(  j+1 1 k)b°  ( j  1  k) ) -i£0 *  ( j  | k)K ( j+1 ! k)  ©  ( j  [ k) 


i°  ( j  i  k)* 


.2°  ( j  i  k). 


-h  (jik)d'  (j+1) 


i°  ( j  I  k) 


£°  (jjk 


(6.4.35) 


We  have  thus  shows  that  if  the  solution  of  (6.4.31)  exists  and  unique, 
the  open-loop  optimal  control  must  be  given  by  (6.4.35);  and  the  O.L.F.O. 
control  is  given  by  (6.3.41).  We  shall  consider  the  question  of  existence 
and  uniqueness  of  O.L.F.O.  control  in  the  next  section. 

6.5  Existence  and  Uniqueness  of  O.L.F.O.  Control 

From  equation  (6.3.41),  we  see  that  if  the  optimal  open  loop  control 
{u°(j;k)}^  X  exists  and  is  unique  for  all  k  =  0,  1,  . ..,  N  -  1,  we  can 

J=K 

conclude  that  the  O.L.F.O.  control  {u  (k)}^_X  exists  and  is  unique.  If  the 

K—O 

solution  of  (6.3.33),  i.e.,  the  matrix  K(j |k) ,  exists  and  is  unique,  then 


the  control  law  given  by  (6.3.32)  and  (6.3.33)  is  the  unique  globally  optimal 
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open  loop  control.  Since  fj(j|k)  is  an  indefinite  matrix,  the  solution  of 
(5.3.33),  K(j|k)  (if  it  exists),  is  not  necessary  nonnegative  definite;  in 
fact  it  is  always  indefinite.  Therefore,  we  cannot  a  priori  conclude  that 
h ( j | k)  +  b°  ‘ ( j | k)K(  j+1 { k)b° ( j  j  k)  will  always  be  nonzero,  and  thus  deduce 
that  K  (  j | k)  will  remain  bounded  in  finite  time.  In  this  section,  we  shall 
establish  the  existence  and  uniqueness  of  the  solution  of  (6.3.33),  K(j’k), 
for  the  case  where  the  terminal  time  is  finite  (N  <  =») ;  this  result  will 
then  be  used  to  prove  the  existence  and  uniqueness  of  the  O. L.  F.  O.  control. 

Let  us  define 


L° ( j | k)  =  < x° ( j | k) ,N  (  j ) x° ( j | k)>+2u° ( j ( k) < 


x°  (j  |k)' 


L°(J|kl 


,d(j+l)>+h(j|k)  (u°(j|k))2 
(6.5.1) 

Lemma  6.5.1:  If  h(£ |k)  +  6°'  (£|k)K(£+l|k)b°U|k)  is  nonzero,  £  =  j,  j  +  1, 
. . . ,  N  -  1 ,  then 


L° ( j  i k)  =< 


'x°(j|k7 

-x°(j|k7 

!°(j+i|k)' 

"x°  (j+1  jk)” 

♦  *.  • 

»  K(j|k) 

»  «  • 

>-< 

•  •  • 

,  K(j+l|k) 

•  •  • 

_£°  ( i  1  k). 

-£°(j|k). 

_o°  (j+1  |k)j 

o°  (j+i  i  k) 

(6.5.2) 


.Proof.  Using  (6.3.32),  (6.3.33),  (6.3.35),  and  (6.5.1),  we  have 


L°(j  |k)  =< 


*x°(j|k)- 

'*°(j|k)- 

'£°(j|k)- 

*  •  • 

>  £(j  |k) 

•  •  • 

>+< 

•  •  • 

-£°  (j  1  k). 

.£°  (j  |k)_ 

_o°  ( j  i  k)_ 

,®  ’  a  I  k)  K( j+l  I  k)£°  ( j  I  k) 

L £°  ( J  I  k)J  1_£°  ( j  I  k)J 

(h  ( j  j  k)+b°  ’  ( j  |  k)K(j+l  |  k)b°  ( j  |  k)  ) _1h  ( j  |  lc)  (h  ( j  |  k)+b°  ’  (j  j  k) K ( j+1 !  k)  • 

rk°(j|k)l 


b0,(j|k)K(j+l|k)©  (j|k) 


U°  (j  |k). 


(6.5.3) 
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Bv  (6. 4. 34),  (6.4.28),  and  (6.4.31),  equation  (6.5.3)  becomes 

'i°(j!k)i  rx°(j|k)i  rx°(j+i!k)-]  fi°(j+ilk)' 

L°(j|k)  =<  ...  ,  K(.i|k)  ...  >-<  ...  ,  K(j+ljk)  ...  ; 

o°(jik).  -0'*1"  — -o,..,.... 


£  (j|k)J  [o°(j+l|k)j 


5+1 }k)  ...  ; 

£°(j+l|k). 


(6.5.4) 


Lemma  6.5.2:  For  all  i  =  k,  k  +  1,  _ ,  S  -  1  «e  have 


A*  A  »  X 

tr{F  ^(slk)  ^  W(3)£^.<3  jlc)V  =  tr|S(i)j^(i|k)  +  (S(j  +  l)R(j)  + 


-x(j|k)1 

2u(j)<  ...  ,  d(j+l)>+u2(j)S(jTi)Zb(j){ 

£(  j  i  k)J  " 


(6.5.5) 


where  u(j),  j  =  i,  i  +  1,  ...,  N  -  1,  is  an  arbitrary  control  sequence  and 


x(i  K) 


l(j  k). 


is  the  resulting  trajectory. 

Proof :  Using  (6.3.28),  we  have  (W(N)  k  ¥) 

tr(W(j+l)I^(j+l!k))  =  tr{A’  (j)K(j+L)A(j)^(j  jk)+u(j)W(j+l)j^(j  ik)+ 
W(j!-l)R(j)+2u(j)W(j+l)A(j)Z_x.Q(j  jk)|  ;  j=i,  i  +  1,  ...,N-1  (6.5.6) 

By  applying  (6.3.28)  repescedly,  (6.5.6)  yields 

j 

t r (W(  j-f  1)  2^  (  j+1  j k) )  =  tr{o^(j  ,i)W(j+l)iA(j  ,i)Z^(i|lc)+y^(2^(j  ,i.+l)W(j+3) 

Z=i 

cA(j,;.+l)R(iO-r2u(i)^(j,£+].)W(j+l)oA(j,Z+l)A(L)4xb(i|k)+u2(L)5^(j,i+l)W(j+l) 


— A^'  ’ (<• ' k)  )} 


(6.5.7) 
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From  (6.3*19),  we  have  for  all  j  <  N  -  1  that 

N-l 

S(j)  =  j£(H-l,j)F  £A(N-l,j)  +y]  o;(M,j)K(y.)^(4-l,j)  .  (6.5.8) 

£=j 

Summing  (6.5.7)  over  j  =  i,  i  4-  1,  ....  N  -  1,  end  using  (6.5.8)  and  (6.3.36). 
we  obtain  (6.5.5)  after  a  fair  amount  of  straightforward  manin  ilation. 

To  describe  the  performance  of  the  optimal  open-loop  control  sequence, 
we  shall  introduce  the  notion  of  "conditional  open  loop  optimal  cost  to  go." 
Definition  6.5.3:  The  conditional  open-loop  optimal  cost  to  go  for  the  dc- 
-erministic  control  problem,  (6.3.26)-(6.3.31) ; 


Ji!k(-(iik),£(ilk>)  =  rai«  7  x’ (N|k)F  x(N!k)4-trF  E(Njk) 

u(j)  :j=i»  •  •  •  ,N-1 


N-l 

+  y^1-  0  lk)W(j |kHtr  W(j)E(j  |k)4-h(j)u2(j)  ]  (6.5.9) 

j=i 

where  x(i|k),  ,I(i|k)  satisfy  the  set  of  equations  (6.3.26)-(6.3.28) . 

Note  that  J?i.  (•,♦)  is  defined  as  a  function  on  M,_  .  .  *  Rn.  From 

(2n)*(2n) 

(6.3.28),  we  see  that  E(jik)  10,  j  *  i,  i  +  1,  ...»  N  -  1,  if  and  only  if 
I^(ijk)  £  0.  Thus,  from  (6.5.9)  we  have 


J,  |k(I»&)  ^  0  if  l  >  0 


(6.5.10) 


By  lemma  6.5.1  and  lemma  6.5.2  we  immediately  deduce: 

Theorem  6.5.4:  If  h(£|k)  +  b°*U|k)K(m|k)b°(i|k)  is  nonzero,  i  -  i,  i  4-  1, 
...,  N  -  1,  then  the  conditional  open-loop  optimal  cost  to  go  has  the  closed 
fora 

N-l 


=  j  trj^SU)  4-^P  S(j4-l)_R(j)[  4-  | 


XT 


(A 


,  i(i|k) 


> 


(6.5.11) 


where 


4-  '4  aid-  ;  4,-14  Old...  ;  2- 


4a 


L  -  ,  e 
— xbrTj 


•  (6.5.12) 


He  Shali  “°“  ,take  “se  °f  (6.5.10)  to  establish  the  esis- 

tence,  uniqueness  and  boundedness  of  K(j|k),  j  =  k,  k  +  1  N  . 

k  =  0,  1,  ...,  s  -  i. 

iSaJLSo:  Let  £(i)H'(j)  1  B,  for  all  B  1  0,  j  .  k,  ....  „  .  j.  Ihe„ 

we  have  S(i|k)  +  £°'  (j  )k)K( j+1  f k)S“ (J  |»  >  0,  j  -  k . „  .  x. 

Proof;  If  F  >  0,  then  since  h(N  -  1)  >  05  we  have 

h(»-l|k)+b0'(S-l|k)K(S|k)b°«-l|k)  -  h(S-l)+tr(^(S-l|k;F) 

+b'’,(M-l|k)F  6°(SI-l|k)  >  0  .  (6.5.13) 

Sow  assume  that  h(t|M  +  6.’ 0 (I ! k>K(JE+l | fc)6° ( fc)  >  0  for  1  -  1  i  +  1 

N  -  1  (k  <  i).  Consider  the  special  case:  R(j)  =  0,  j  -  k,  ....  N;  then, 

by  the  induction  hypothesis,  theorem  6.5.4  and  (6.5.10)  imply  that 

Ji I k <ACi-l i k)  , b° (i-1  ? k)  =  r.r(^(i-ijl0S(i))+b*°(i-l|k)K(i|k)bo(i-l|^  >  0 


where  we  have  chosen 


(6.5.14) 


Z(i-l'k)  = 


4°(i-lfk) 


j  zJ(i-l!k)G'(i-l)' 


G(i-i)zb(i+i|k)  :  zj(i-i|k) 

^(i-lik)  :  z£(i-l|k)G'(i-l) 


G(i-l)Zb(i-l|k)  :  G(i-l))Z?(i-i jk)G' (i— ix! 


(6.5.15) 
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and  since  h(i  -  1)  >  0,  we  have  from  (6.3.37)  that 

h  (i-1 1 k)+b° 1 (i-1 j  k)K(i | k)b° (i-1 j  k)  =  h(i-l)+J°|k(|(i-l|k) ,b°(i-l|k))  *  0  . 

(6.5.16) 

Thus  the  lemma  is  proved  by  induction. 

We  can  now  easily  prove  the  existence  and  uniqueness  of  the  solution  of 

(6.3.33),  (K(j|k)}N:J,  k  =  0,  1,  ....  N  -  1. 

3-K- 

Theorem  6.5.6:  (Existence  and  Uniqueness)  Let  1J  £  G [( j )  15  £’  ( j )  ,  j  =*  0,  ..., 
N  -  1  f--r  all  B  >0.  The  solution  of  (6.3.33),  (K(j  |k) exists,  is 

~  J“K 

unique  and  is  bounded,  (N  <  °°) ,  if  f^(kjk),  b°(k|k)»  A(k) ,  W(k) ,  £  N(k)  and 
h(k)  are  bounded,  k  =  0,  1,  . ..,  N  -  1. 

Proof:  The  equation  (6.3.33)  can  be  written  as  a  set  of  two  equations  (see 
chapter  2,  section  2.5) 

K(j|k)  =  (©  (jjk)-b°(j|k)Vo(j+lik))’K(j+lik)(0  (j|k)-b° (jlk)^ (j+1)) 

+D(j|k)+V^(j+i)h(j|k)Vo(j+l|k)  ;  K(n|k)  =  F  ;  j  -  k,k+l . N-l 

k  =  0,1,..., N-l 

(6.5.17) 

V,(j+l|k)  =  (h(j|k)+b0,(j|k)K(j+l|k)b°(j|k))''1£0,(j|k)K(j+l|k)®  (j|k) 

(6.5.18) 

From  (6.3.38),  (6.3.40)  and  the  assumption  on  boundedness,  we  have  that 
b° ( j | k)  and  h(jjk)  is  bounded  for  j  =  k,  k  +  J>  ...»  N-l;k=0,  1,  ..., 
N-l.  By  lemma  6.5.5  V^j+ljk)  exists,  is  unique  and  is  bounded.  The 
assertion  follows  from  the  linearity  of  (6.5.18)  and  the  fact  that  N  <  °°. 


Corollary  6.5.7:  Ha  optimal  oner -loop  control,  ’ u° (3  k) 
jaiqog  ana  is  bcscsc  if  the  assrsotioas  in  ineorsa  0.3.6 
furthermore ,  x(k  k,C  (0,k— 1)),  ^r{k;k,U  (0,fc— 1)),  b(k  k,i 
bocuced- 


X-i 

•i=k>  exists, 
are  satisfied 
(0,k-l>>  are 


is 


Theorem  6.5.S:  Let  B  -  GC)3C'(jJ,  j  =  0,  1,  ....  X  -  1,  ior  all  B  i  0, 
and  A(k>,  S(k),  X(k),  G(k),  5£(k),  h(k),  ?  are  bounded,  k  =  0,  1,  ....  X  -  1; 

tie  O.L.F.O.  Central,  c  (k) ,  k  *  0,  i.  — ,  X  -  1,  exists,  is  unique  and 
is  bounded - 


Proof:  Ve  sh?1l  1  use  induction  on  k.  Wses  k  =  0,  s(0  0)  and  b(0  0)  are 

bounded  airisi  surely;  also  2(0  0)  is  bounded;  thus  b~  corollary  6.5.7, 

* 

a°(9  0)  exists,  is  unique  and  is  bounded  a.s.  By  (6. 3-*l),  u  (0)  exists, 
is  unicue  and  is  bounded.  Assure  the  statement  of  the  tneoren  is  true  *.or 
k  =  0,  ...»  i;  1  <  S  -  1-  By  the  assumptions  and  the  induction  hypothesis. 


* 


the  identification  equations,  {6.3.19)— (6.3.23),  imply  that  x(*-3-l  *+l.l'  {0,  • )  r 
and  b(2rl  4fl,i‘  (0,*.))  are  bounded  a.s.s  and  that  2(1+1  *.-=-1,1*  (0, <-))  is 
bounced,  thus  corollary  6.3./  implies  that  u  ( ^1 .  *-"*"1)  exists,  is  unique 
and  is  bounded  a.s.;  by  (6.3.41)  the  assertion  of  the  theorem  holds  for 
u* (k) ,  k  =  0,  1,  ...,  if  1. 

One  would  like  to  extend  the  results  to  the  infinite  tine  case  with 
X  -  Unfortunately,  this  is  seldoaly  possible.  From  (6.3.39),  we  note 


that  if  we  let  X  -  x,  S(j)  will  remain  bounded  if  and  only  if  A(k)  is  ex¬ 
ponentially  stable;  thus,  the  solution  of  (6.3.33)  t  jy(  j  k) ,  with  X  •-  *  will 
not  be  meaningful  unless  A(k)  is  asymptotically  stable.  In  many  cases  of 


interest,  the  system  to  be  controlled  is  unstable.  Therefore,  we  shall  not 
investigate  the  solution  of  (6.3.33)  with  X  —  *■. 
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6.6  Asynptotic  Behavior  of  the  Identifier 

In  this  section,  we  shall  study  the  asynptotic  behavior  of  the  identi¬ 
fication  equations.  The  results  will  allow  us  to  consider  the  problem  of 
controlling  the  system  §  over  an  infinite  time  interval  (N  -»•  °») . 

OO 

Definition  6.6.1:  {  (A(k) ,  £(k) )  J^q  is  said  to  be  completely  observable 

of  index  v  at  k  if  the  observation  matrix 


M,  _(k,v)  =  [C*  (k)  I  ^;(k,k)C'(k  +  1)  ...  I  i!(k  +  v  -  2,k)C’(k  +  v  -  1)] 

•  n  •  — H. 

(6.6.1) 

is  of  full  rank  n.  i (A(k) , C (k) ) is  said  to  be  uniformly  completely 
observable  of  index  v  if  the  pair  is  completely  observable  cf  index  v  for 
all  k  =  Oj  1,  ...  . 

as 

Theorem  6.6.2:  Let  t (A(k) , C(k) ) be  uniformly  completely  observable  of 
index  v,  and  suppose  that  A(k),  G(k)  are  nonsingular,  k  =  0,  1,  ...  .  If 
u(k)  /  0,  k  =  0,  1,  ...,  then  { (A(k,u(k)) ,C(k) is  uniformly  completely 
observable  of  index  v',  v*  1  2v. 

Proof:  By  (6.3.17)  and  (6.3.20),  we  have 


Mi  -(k,2v)  = 

A  y 


C(k) 


C(k+l)^A(k,k) 


:  o 


C(k+l)u(k) 


k+j+1  • 

c(k+j)bA(k+j-i,k)  :  £  c(k+j)6A(k+j-i,m)u(i)^G(£-i,k) 


i=k 
k+2v-2  : 


C(k+2v-l)i  (k+2v-2,k):  £  C(k+2v-lU  (k+2v-2,il+l)u(£U  (£-l,k) 


£=k 


(6.6.2) 


By  assumption,  the  first  mv  rows  of  vectors  contains  at  least  n  independent 
vectors.  Among  the  rows  vectors  C(k  +  v  +  j)£.(k  +  v  +  j  -  l,k),  let 

A 
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c'  .....  (k  +  v  +  j)6.(k  +  v  +  j  -  l,k),  ...  cj  ,  .(k  +  v  +  j)* (k  +  v  +  j  -  l>k) , 

be  the  v .  vectors  which  are  independent  of  the  row  vectors : 


3 

C(k  +  v)o. (k  +  v  -  ljk) ,  C(k  +  v  -  l)£,(k  +  v,k),  ...  C(k  +  v  +  j  -  1) 
—  A  A 

e. (k  +  v  +  j  -  2,k),  j  =  1,  ...»  v  -  1;  where 
A 


C(k  +  v  +  j) 


'^(k  +  v  +  j) 
c’  (k  +  v  +  j). 

**“  — m 


(6.6.3) 


and  c .  ( • )  is  some  permutation  of  {1,  2,  ...»  m}.  Since  (A(k)  ,C(k))  }^_g 
is  uniformly  completely  observable  of  index  v,  it  follows  that  v^  ?  0, 
i  =  1,  ....  v  -  1,  and  ti  at 


m  +  v.  +  V-  +  ...  v  ,  =  n  .  (6.6,4) 

1  2  v-1 

Assume  that  we  have  the  dependence 

v+j-1 

C’  (s ) (k+v+j ) cA (k+v+j - 1 ,  k)  =  ^  a^(j,s)C(k+i)^A(k+i-ltk);  H  s  i  v.  (6.6.5) 

j  i=0 

where  the  only  possible  nonzero  entiles  of  o^(j,s),  i  =  0,  . ..,  v  +  j  -  1,  are 

those  corresponding  to  independent  rows  of  jC(k  +  i)0,^(k  +  i  -  l»k),  i  -  0, 

. ..,  v  +  j  -  1.  If  there  exists  no  a^(j,s),  i  =  0,  ...,  v  +  j  -  1,  which 

bears  the  relation  (6.6.5),  then  the  (m(v  +  j  -  1)  +  o(s))th  row  vector  of 

Mi  ~(k,2  0  is  independent  of  the  first  m(v  +  j  -  1)  row  vectors.  If  there 
*”A  f  C 

exists  a! (j ,s) ,  i  =  0,  ...,  v+j-1  which  gives  the  dependence  (6.6.5), 
then  such  a  dependence  is  unique  by  construction.  Now  assume  that  the 
(m(^  +  j  -  1)  +  c(s))th  row  vector  of  Mi  ^(k,2v)  is  dependen"  on  the  first 
m(,  +  j  -  1)  row  vectors,  then  we  must  also  have  the  dependence 
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k+v+j-1 

53  ^  (s)(k+\H-j)*A(k+v-M-l,£+l)uUJ^G(5-l,k)  = 

£=k  j 

v+j-1  k+s-1 

S£i(j,s)53  -(k+i)^A (k+i-1 * *+D  uODj^a-l  ,k)  •  (6.6.6) 

i=l  £-k 

Since  A(k)  is  nonsingular,  by  (6.6.5)  we  have 
r+j-1  k+v+j-1 

^  53  aJCj,s)C(fcH.)^(kfi-J,£+l)u(t)^;(4-l>k)  -  0^  (6.6.7) 

i=0  i=k+i 

where 

ji;A(i,j)  =  A  1(i)A  ^(i  +  1)  ...  A  1(j)  ;  i  >  j  .  (6.6.8) 

oo 

Since  { (A(k),C(k))}^_G  is  uniformly  completely  observable,  the  vector 

£Lj'(j,s)  4  [o' (j,s)  ...  I  o!(j,s)]  (6.6.9) 

o  . 

cannot  be  the  zero  row  vector,  s  **  1,  ...»  v^.  3y  assumption  G(k)  is 
nonsingular,  therefore  (6.6.7)  is  true  if  and  only  if  u(k  +  i)  =0,  i  *  0, 

1,  ...,  j  which  is  a  contradiction.  This  result  applies  for  s  =1,  ...»  v.; 

•  J 

j  =  0,  1,  ...,  v  -  1.  Together  with  (6.6.4)  and  the  remark  made  at  the 

beginning  of  the  proof,  we  have  that  Mr  ~(k,2v)  will  have  rank  2n  if 

•~A,C 

u(k  +  i)  ^  0,  i  =  0,  1,  ...,  v  -  1.  The  theorem  follows  from  the  assumption 
that  u(k)  ^  0,  k  -  0,  1,  ...  . 

Corollary  6.6.3:  Let  A(k),  G(k)  be  bounded  and  nonsingular.  If 

oo 

{ (A(k)  ,£(k)) is  uniformly  completely  observable  of  index  v,  the  error 
covariance  matrix,  £(k|k,U(0,k-l))  which  satisfies  (6.3.21)  to  (6.3.23),  will 
remain  bounded  for  all  k  =  0,  1,  ...  where  u(k)  is  any  bounded  but  nonzero 
control  for  all  k  =  0,  1,  ...  . 
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Lemma  6.6.4:  Suppose  that  £(k)  satisfies 


G(k)B  G'(k)  <  B  ;  BeM  ,  BiO  .  (6.6.10) 

— -  —  —  tin  —  — 

Let  Y(k)  =  (),  i.e.,  there  is  no  driving  noise  in  gain  dynamics;  then  for 
any  control  sequence,  we  have 


(k+1 1 k+l , U(0 ,k) )  <  ^(kfk.UCOik)) 


Proof:  From  (6.3.23)  and  (6.3.21),  since  N(k)  =  (J,  we  have 


(6.6.11) 


Mk+l|k+l,U(0,k))  *  G{k)L(k!k,U(0,k-l)G'(kHO:i  )V  (k+l|k,U{0,k)) 


(C(k+l)A(k|k,U(0,k))C’  (k+l)+£(k+l))V  *  (k+l|k,U(0,k))  . . . 

I 


(6.6.12) 


where  V  (k+l|k,U(0,k))  satisfies  (6.3.21)-{6.3.23) ,  using  (6.6.10),  (6.6.11) 
follows  immediately  from  (6.6.12). 

An  immediate  consequence  of  lemma  6.6.4  is  that  if  (6.6.10)  is  true  and 
X(k)  =  J),  then  there  exists  E  such  that 


lim  E,  (k(k,U(0,k  -  1))  =  Z. 


(6.6.13) 


Mote  that  (6.6.13)  is  true  independent  of  the  observability  of  { (A(k) ,C(k)) 
In  the  following  theorem,  we  shall  give  sufficient  conditions  under  which 
Z,  =0. 

— D  — 

Theorem  6.6.5:  Let  X(k)  -  £,  A(k) ,  G(k)  be  bounded  and  nonsingular  and 
.G(k)  satisfies  (6.6.10),  k  =  0,  1,  ...  .  If  { (A(k) ,£(k))}^_Q  is  uniformly 
completely  observable  of  index  v  and  u(k)  is  any  bounded  but  nonzero  control 
for  k  =  0,  1,  ...»  then 


lim  Z^(k|k,U(0,k  -  1))  =  0 


(6.6.14) 
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Proof:  Let  e  >  0  sties  that 

I  i4(k+2v|k+2v,U(0,k+2v-l))-  4(k|k,U(0,k-l))i|  it  (6.6.15) 

where  ]|*J|  is  the  spectral  norm.  Since  I^(k.jk,U(0,k-l))  i  0,  k  *  0,  1, 

. ..,  (6.6.11)  and  (6.6.15)  imply  that  we  have  the  inequality 

|l4(k+j|k+j,U(0,k+j-l)  -  Zb(k+j-l|k+j-l,U(0,k+j-2)|i  i  .. 

j  “  1,  2,  ...»  2v  .  (6.6.16) 


Using  equation  (6.6.12),  we  have 

e  2  j  1  ro:iJV*(k+j  |  k+ j -1 , U (0 , k+j -1 )  •  (C  (k+ j ) £(k+j -1  j k+ j -1 , U (0 , k+j -1 )  ; • 

C' (k+j)  +  ^(k+j))V*’ (k+j  j k+j-1 , U (0,k+j-l) ) 

By  corollary  (6.6.3),  £(k+j)&^(k+j-l|k+j-l,U(0,k+j-l))C^' (k+j)  +  ^.(k+j)  can 
he  uniformly  bounded,  so 


(6.6.17) 


f  I  (C  (k+j  )  Mk+j  -1 1  k+j  -1 ,  U  (0 ,  k+j  -1 )  )  C\k+j  )+5.(k+j  )  )  V* '  (k+j  j  k+j -1,U(0, k+j-1)) 


1  J 


!  (C(k+j)A(k+j-l|k+j-l,U(0,k+j-l)CXk+j)+2(k+j)f  i  ;  ♦  I  I  (C(k+j  )|(k+j-l  j  k+j-1  ,U(0,  k+j-])  ) 


C'  (k+j )+g(k+j ))V (k+j  | k+j -1 , U (0 , k+j -1) ) 


0 

i  j 


S  C  /e  4  i  fe)  j  -  1,  2,  ....  2v 


(6.6.18) 


6^(e)  is  continuous  in  e  and  6^  (e)  -*•  0  as  z  ■+  0,  j  =  1 ,  ...,  v.  Using 
(6.3.21),  (6.6.18)  can  also  be  written  as  follows 


j :  C  (k+j ) A(k-f j-DZ^ (k+j-1  |k+j-l,U(0,fc+i-2})G’  (k+j-l)+u(k+j-l)C  (k+j) • 


I  (k+j-1 ' k+j -1  ,U (0  ,k+j~3  (k+j-l)  \  \  <  5  .(c) 

— n  J 


=  1 


»  -  -  •  >  -<• 


(6.6.19) 


Since  V  (k+j  j k+j -1,11(0, k+j-i))  is  bounded  for  j  *  1, 
(6.6.17)  and  (6.3.21)  imply  char 


tnererore 


!  I  f0»x  JV  (k+j  !k+j-l,U(0,k+j-l))C(k+j)£(k+j-l!k+j-l,U<0,k+j-l)) 


i  iS,W 


(6.6.20) 


\  |  [1  :o]V  (k+j  j  k+j -1 , U (0 ,k+j -1)  ) C (k+j  ) A (k+j-1  j  k4j -1  ,U  (0 ,k+j -1)  ) 

n . -  1 

*  S,(e) 


•  «  • 
I 


(6.6.21) 


where  S^(s)  is  continuous  in  £,  £^(e)  —  0  as  £  -*■  0,  i  *  1,  2,  ...  .  3v  using 


(6.3.23),  (6.6.20)  and  (6.6.21)  and  the  assumption  that  G(k)  is  nonsingular, 
the  inequality  (6.6.19)  implies 


I : [c(k+i  :  0] 


I  ..  (k+1 | k+1, U (0,3) 


— XD 


i!<f1(s)  (6.6.22) 


E.  (k+1 1 k+1  ,U(0,k)  )J 

— D 

k+j-1 

1  [£ (k+j ) 4^ (k+j-1 , k+1)  :c(k+j)  ^  ^  6A(k+j-2,£+l;u(i)2G(i-i,k+l)] 

Z^k+l 


(k+1  j  k+1  ,U  (0  ,k))' 


Z,  (k+l|k+l,U(0,k))  J 


I!  5  f^g) 


j  =  2,  3,  ...,  2v 


(6.6.23) 
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where  f^(s)  is  continuous  in  e,  f^(s)  ^  0  as  £  -*■  0,  i=  1,  2,  . ..,  2\>. 
Equations  (6.6.22)  and  (6.6.23)  imply 


!  I  — A,C(it+1’2v)! 


,  (k+1 1 k+1 ,  U  (0 ,  k)) 
— xb 


Z.  (k+lik+l,U(0,k))  J 


^b 


!  i  ^  f(e) 


(6.6.24) 


where  f(e)  -*•  0  when  e  ■*  0  and  is  continuous  in  e.  By  theorem  6.6.2, 

Hr  r(k  +  l,2v)  is  of  full  rank,  so  we  have 
— A,C 


l|l  (k+1 jk+l,U(0,k)) | |  <  S'(e)  6’(e)  -  0  as  e  -  0  (6.6.25) 

— xb 

i  |£  (k+ljk+l,U(0,k))|  |  <  o"(e)  o"(e)  -  0  as  e  -  0  .  (6.6.26) 


Now  the  conclusion  of  the  theorem  follows  from  (6.6.13). 

Theorem  6.6.5  can  be  extended  to  the  case  where  u(k)  is  bounded  but 
nonzero  control  for  all  but  a  finite  number  of  k*s.  Since  jZ(kjk,U(0,k-l))  2 
(6.6.14)  also  implies 


lim  Z  .  (kik,U(0,k-l))  *  0  (6.6.27) 

,  ~ XD  — 

if  the  conditions  for  theorem  6.6.5  hold. 


Let  us  consider  an  observable  systemS,  (6.2.1),  the  gain  parameters 
are  assumed  to  be  unknown  and  satisfy 

b(k  +  1)  =  G(k)b(k)  (6.6.28) 

with  G(k)  satisfying  (6.6.10).  Assume  that  we  want  to  control  the  system  S 
over  an  interval  N  <  ®.  In  the  beginning,  the  modified  weighting  on  the 
control  is  high,  and  thus  in  general,  the  control  magnitude  will  be  low  at 


the  beginning.  Thus,  the  trajectory  of  the  overall  control  system  would 
be  pretty  much  the  same  as  the  input-free  trajectory  of  the  system  S.  If 
the  matrix  A(k)  is  exponentially  stable,  the  true  state  of  the  system  will 

evolve  toward  zero  by  using  negligibly  small  control  magnitudes  (even  zero) . 

%N— 1 

The  result  is  that  little  effort  of  the  input,  (u(k)/^_Q  is  spent  for  control 
and  identification  purposes.  We  would  expect  that  the  estimated  parameters 
will  hardly  converge  to  the  true  parameters,  b(k) .  On  the  other  hand  if 
A(k)  is  not  exponentially  stable,  then  the  true  state  of  the  overall  system 
will  diverge.  This  diverging  phenomenon  will  be  noticed  by  the  identifier, 
thus  resulting  in  a  high  control  magnitude  because  of  (6.3.32).  Since 
little  is  initially  known  about  the  gain  parameters,  the  high  magnitude  con¬ 
trol  will  be  utilized  mainly  for  identification  purposes.  Therefore  the 
control  will  be  kept  bounded  away  from  zero  as  long  as  exact  identification 
of  Jb (k)  has  not  been  obtained.  Using  theorem  6-6.5,  we  predict  that  the 
estimated  parameters  of  _b(k)  will  converge  to  the  true  gain  parameters 
before  the  control  magnitude  goes  to  zero. 

Analytical  studies  of  the  convergence  rate  of  the  O.L.F.O.  system  are 
not  yet  available.  From  the  above  discussion,  we  may  predict  roughly  that 
the  convergence- rate  for  unstable  system  will  be  relatively  fast  depending 
on  "how  stable"  the  system  is;  and  the  convergence-rate  for  stable  system 
will  be  very  slow. 

For  control  over  an  infinite  time  period,  see  section  6.7(C)  for  de¬ 
tailed  discussions. 

Finally,  we  shall  discuss  some  interesting  implications  of  theorem 
6.6.5.  Consider  an  observable  system  S,  (6.2.1),  with  unknown  gain 


parameters  satisfying  (6.6.28)  and  with  G(k)  satisfies  (6.6.10).  Let 

o,  (x(k|k,b(k|k) ,E_  (k|k,U(0,k-l))  be  any  a-i-fccc  control  law  which  is  "put" 
k.  d 

after  the  identifier  (see  Figure  6.4)  and  with  the  following  properties 
(k  ±  0) : 

2>  4(«,b,0  *  0.  x«Rn.btR".rC!^,x#05E_*0 

3)  s.(x,b,0)  -  -(h(k}+b,0t)K(k+l)b(k))*1b'(k)K(k+l)A0c)it  ; 


From  condition  2,  we  see  that  Z^(k[k,U(0tk-l))  •»  0  as  k  "  (6.6.14);  and 

so  from  condition  3,  the  ad-hoc  control  scheme  will  converge  to  the  optimal 

control  strategy  when  the  full  dynamics  become  known.  This  indicates  thar 

the  ad-hoc  scheme  c,  (x(kjk) ,b(k|k) ,Z,  (klk»U(Q,k-l))  can  provide  reasonable 
— K  —  —  — D 

simulation  results. 


6.7  General  Discussion 

In  this  chapter,  we  investigated  the  problem  of  identification  and 
control  of  discrete  linear  systems  with  unknown  gain  from  the  theoretical 
standpoint.  The  control  is  open-loop  feedback  optimal.  The  implementation 
of  such  a  control  (O.L.F.Q.  control)  was  described  to  some  detail.  The 
actual  implementation  for  O.L.F.O.  control  for  third-order  systems  will  be 
discussed  in  more  detail  in  chapter  7.  As  we  shall  see  later,  such  a 
proposed  scheme  appears  to  be  computationally  feasible  and  that  the  results 
are  reasonable  and  appealing.  A  deeper  theoretical  understanding  of  the 
derived  O.L.F.O,  control  is  possible  from  the  results  in  sections  6.4  and 
6.5.  The  questions  of  existence,  uniqueness  of  O.L.F.O.  control  are  con¬ 
sidered  in  great  detail.  The  asymptotic  behavior  of  such  control  systems 
was  treated  in  section  6.6;  some  of  its  extensions  wj',1  be  discussed  later 


in  this  section. 


Fir.  h. 4  BLOCK  DIAGRAM  FOR  AD-HOC  CONTROL  SYSTEM 
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(A)  Discussion  on  Approaches 

The  problem  of  combining  identification  and  control  of  linear  system 
with  unknown  gain  have  been  considered  by  several  people.  Farison  [60] 
considered  an  ad-hoc  procedure  which  basically  assumes  the  separation  be¬ 
tween  identification  and  control.  Murphy  [61]  considered  the  approximate 
effect  of  iteration  between  control  and  identification  and  he  pre-supposed 
that  the  control  was  a  pure  feedback  of  the  estimated  spates.  Gorman  and 
Zaborszky  [62]  used  a  similar  approach  to  that  of  Murphy  and  obtained  a 
suboptimal  control  which  required  the  solution  of  a  sequence  of  two  point 
boundary  value  problems.  Essentially,  [61]  and  [62]  are  approximately 
Bellman's  equation.  The  approach  taken  in  this  chapter  is  different  from 
those  in  [60],  [61],  and  [62]. 

Bar-Shalom  and  Sivan*  [63]  also  used  the  O.L.F.O.  control  approach  to 
consider  control  problems  with  random  parameters.  They  derived  a  general 
solution  but  made  no  attempt  to  study  analytically  the  derived  results.  The 
approach  taken  in  this  chapter  is  primarily  motivated  by  computational 
feasibility 

From  the  discussion  made  at  the  end  of  section  6.6,  we  can  see  why 
different  computation  schemes  suggested  by  Farison,  Murphy,  Gorman  and 
Zabor.'.zky  will  all  be  expected  to  give  reasonable  simulation  results.  It 
is  ha’-d  to  quantitatively  compare  our  approach  with  theirs  without  extensive 
simulation  experiments.  One  computation  advantage  of  our  results  over  those 


This  reference  was  brought  to  the  author's  attention  when  most  of  the 
theoretical  work  of  this  chapter  (sections  6.2  to  6.5)  had  been  completed. 


of  Murphy  a  Gossan  and  Zahars  iky  is  that  ve  replace  a  sequence  of  cvo  point 
boundary  problems  by  solving  a  sequence  of  matrix  Siccaci  type  difference 
equation  of  dissension  (u  f  l)n  *  (n  ■*  l)e,  (6-3.33).  this  matrix  Eiccati 
difference  equation  is  solved  backward  in  time  starting  frctii  the  terminal 
tinfi  to  tbe  present  time  k;  i .e. ,  aaS  -  k  steps  computation,  where  k  =  G, 

1,  — j,  S  -  1.  Computational  vise,  this  is  easier  than  solving  a  tvo  point 
boundary  problem.  In  our  approach,  tbe  theoretical  proof  on  the  existence 
and  uniqueness  of  O.L.F.O.  control  sequence  is  available;  this  gives  us 
confidence  in  trying  out  the  subop tins!  control  scheme  using  a  digital 
computer.  Also,  ve  can  deduce  and  predict  roughly  the  behavior  of  the  over¬ 
all  O.L.F.O.  control  system  (section  6.6)  from  the  derived  equations  (section 
6.3). 

(3)  Vector  Control 

la  our  investigation,  ve  assumed  thac  the  control  is  scalar.  However, 
the  approach  can  be  extended  in  a  straightforward  conceptual  manner  to  the 
vector  control  case.  First,  a  set  of  identification  equation  is  derived 
which  will  generate  the  estimate  of  the  current  state,  the  current  estimate 
of  the  unknown  gain  matrix  and  the  different  cross-error-covariance  matrices. 
An  open-loop  control  problem  is  formulated  as  in  section  6.3,  equations 
(6.3.20)  to  (6.3.31)  and  discrete  matrix  mininun  principle  is  used  to  obtain 
the  extremal  solution.  The  results  will  be  similar  to  those  of  scalar  con¬ 
trol  case.  However,  the  equations  in  the  vec  or  control  case  will  be  more 
complicated. 

(C)  Control  Over  Infinite  Interval 

Let  us  consider  the  problem  of  controlling  the  system  S,  which  is  tine 
invariant  and  unknown  constant  gain  Ij,  over  an  infinite  interval,  i.e.,  N  -*■  « 
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It  was  pointed  out  (in  section  6-5)  that  the  problem  will  not  be  very 
meaningful  in  many  cases  if  we  just  consider  the  obtained  results  (section 
6.3)  and  let  X  -  We  suggest  the  window-shifting  approach.  Assure  that 
at  ail  tines,  we  have  X  more  steps  to  control  (see  Figure  6.5),  thus  at  all 
tines  we  solve  an  open-loop  control  prohien  over  an  interval  of  X  steps. 

Ibis  approach  is  activated  by  computational  consideration  and  the  theoretical 
results  derived  in  section  6.6. 

We  note  that  in  the  O-L.F.O.  approach,  we  have  to  resolve  the  open- loop 
control  problem  at  all  tine  k  so  as  to  adjust  the  control  scheme  accordingly. 
In  our  case,  we  have  to  corpute  K(k  k)  in  a  backward  direction  starting  from 
the  terminal  time  X  to  k  for  each  k.  If  X  is  very  large,  this  computation 
will  require  a  very  long  tine  to  accomplish.  From  a  computational  stand¬ 
point,  we  would  like  to  "cut  back”  the  terminal  time.  Conceptually,  in 
trying  to  control  over  an  infinite  time  period,  the  controller  looks  into 
all  future  effects  caused  by  present  action,  and  decides  on  the  optimum 
move.  The  window-shifting  approach  suggests  that  instead  of  looking  at  all 
future  effects,  the  controller  looks  at  only  near  future  effects  caused  by 
present  actions  and  decides  on  suboptinal  moves.  One  may  view  such  an 
approach  as  a  "short  terra  adaptive  schsme."  Note  also  that  we  can  adjust  the 
"window  width"  according  to  computational  capability.  At  all  times,  we  need 
only  to  solve  for  K(k|k)  in  a  backward  direction  starting  from  X  +  k  to  k. 
Thus  from  a  conceptual  and  a  computational  point  of  view,  such  an  approach 
may  be  desirable. 

Assume  that  the  tine  invariant  system  S  being  controlled  is  observable 
and  controllable.  If  b  is  known  exactly,  then  if  we  consider  control  over 
infinite  time  period,  the  optimal  feedback  gain  is  constant  and  is  given  by 
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£  =  -(h  +  b'K  b)"lb'K  A 


(6.7.1) 


where  K  is  given  by  the  steady  state  solution  of 

K...  =  A’(K.  -  K.b(h  +  b5K.b)-1b'K.)A  +  w  ;  K  =  F  (6.7.2) 

— l+i  ~  —i  — i—  - i— - 1  —  —  — o  — 

(see  chapters  3  and  4) .  Let  N  be  the  integer  such  that  for  n  >  N, 


!  ’-^n  “  Vl1  1  "  £  ;  £  >  0  *  (6.7.3) 

Such  an  integer  N  can  be  found  experimentally  off-line.  Adjust  the  window 
width  to  equal  to  N,  and  apply  the  window-shifting  approach.  Add  some 
nonzero  control  for  identification  purpose  if  it  is  necessary  (see  also 
chapter  7).  Using  the  results  in  section  6.5,  the  existence  and  uniqueness 
of  such  control  sequence  is  guaranteed.  By  theorem  6.6.5,  the  estimate  in 
b  will  converge  asymptotically ,  and  so  when  b(k!k.U*(0,k  -  1)  -  b,  we  have 


K(kjk) 


K(k,N+k;F)  :  0 


where  K(k,N  +  k;£)  satisfies 

K(k,N+k;F)  =  A’(K(k+l,N+k;F)  -  K(k+l,N+k;F)b(h  +  b'K(k+l,JHfc;F)b)_1* 

b'K(k+l,N+k;F))A  +  W  ;  K(N+k,N+k;F)  =  F  (6.7.4) 

and 

u* (k j k)  £(k)x°(k  k)  =  -(h  +  b,K(k,N+k;F)b)-1b’K(k,N+k;F)A  x°(k  k)  (6.7. 

(See  discussion  at  the  end  of  section  6.3.)  Comparing  (6.7.2)  and  (6.7.4), 


we  note  that 


I 
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K(k,N+k;F)  =  «  K  .  (6.7.6) 

Thus  asymptotically,  we  have  a  time  invariant  overall  control  system. 

(D)  Convergence-Rate 

We  have  not  studied  in  detail  (analytically)  the  convergence-rate 
of  the  suboptimal  O.L.F.O.  control  system.  We  can  only  deduce  and  predict 
some  rough  qualitative  estimates  about  convergence-rate  for  stable  and  un¬ 
stable  systems.  We  shall  study  the  question  of  convergence-rate  via  simula¬ 
tions;  some  conclusions  and  discussion  will  be  included  in  the  next  chapter. 

(E)  Conditions  for  Convergence 

From  theorem  6.6.5,  we  note  that  if  x(k)  =  (),  the  sufficient  conditions 
for  convergence  are  observability,  nonzero  control  and  (6.6.10).  The  first 
two  conditions  are  relatively  easy  to  understand  and  intuitively  appealing. 
The  third  condition  needs  some  explanation. 

Suppose  that  G(k)  satisfies  (6.6.10);  then  by  taking  B  =  I  ,  we  have 

_  _  —ri 

G(k)G’(k)  *  1  .  (6.7.7) 

— n 

Thus,  we  have 

i!G(k)||  <  1  (6.7.8) 

where  jj*||  is  tne  spectral  norm.  Equation  (6.7.8)  provides  us  with  the 
necessary  condition  for  (6.6.10)  to  hold.  Intuitively,  (6.7.8)  means  that 
the  uncertainty  of  t>(k)  cannot  grow. 

Let  G(k)  be  an  nxn  matrix  such  that 

G(k)x  <  x  ;  x  z  Rn  .  (6.7.9) 


This  discussion  was  motivated  from  a  suggestion  made  by  Prof.  J.  C.  Willems. 
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If  B  2  0,  Chen  B  =  DD*  for  some  D  t  M  .  Let  D  =  [d  ...  d_],  equation 

”*  “  — —  tin  “  l 

* 

(6.7.9)  inplies 


G<k)DD’ G' (k)  <  DC’  (6.7.10) 

and  so  G(k)  satisfies  (6.6.10).  Thus  (6.7.9)  provides  a  sufficiency  test 
for  (6. 6. 1C).  Geometrically,  (6.7.9)  inplies  that  (J(k)  is  a  linear  transfer- 
nation  which  is  directionally  invariant  but  shrinking  or  retaining  the 
length  of  each  vector.  Some  weaker  sufficiency  tests  which  have  some 
physical  interpretations  will  be  explored  in  future  research  efforts. 

(F)  Different.  Cost  Criterion 

The  approach  can  be  applied  to  the  more  general  case  where  a  cost 
criterion  other  than  quadratic  is  being  considered.  The  identification 
equations  remain  unchanged  but  tha  open-loo?  control  problem  thus  formu¬ 
lated  will  be  different  from  (6.3.26)-(6.3.31) .  By  using  the  discrete 
matrix  minimum  principle,  we  shall  obtain  a  set  of  equations  which  define 
a  two  point  boundary  values  problem. 

6.8  Perspective 

The  problem  of  stochastic  control  of  linear  systems  with  unknown  gain 
was  also  treated  by  Florentin  [64],  Farison  [60],  Murphy  [61],  Gorman  and 
Zaborszky  [62].  The  approach  in  [61 ]— [64 ]  is  that  of  approximating  the 
solution  of  Bellman's  equation.  [60]  presuppose  separation. 

Open-lcop  feedback  controller  was  described  by  Dreyfus  [26].  Open- 
loop  feedback  optimal  control  approach  was  also  used  by  Bar-Shalom  and 
Sivan  [63]  in  considering  control  of  discrete-time  linear  systems  with  random 


parameters. 
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|  To  the  author’s  knowledge,  for  this  particular  problem  of  controlling 

I  linear  system  with  unknown  gain,  the  investigations  in  sections  6.5  and  6.6 

represent  the  first  extensive  analytical  studies  on  the  derived  suboptimal 

[solution-  The  contributions  being  that  a  plausible  computationally  feasible 
suboptimal  solution  is  derived  using  Che  O.L.F.Q.  approach,  extensive 
b  analytical  studies  on  the  derived  solution  are  carried  out,  and  from  the 

jj  derived  results  some  rough  behavior  of  the  overall  suboptimal  control 

\  .  system  can  be  deduced;  also  we  have  a  deeper  understanding  on  the  effects 


CHAPTER  VII 


CONTROL  OF  THIRD  ORDER  SYSTEMS  WITH  UNKNOWN  ZEROES: 
NUMERICAL  EXAMPLES 


In  the  last  chapter,  re  have  studied  theoretically  the  problea  of 
control  of  a  discrete  tine  linear  systea  with  unknown  gain  under  the 
quadratic  criterion.  A  suboptiaal  adaptive  control  systea  was  derived 
using  the  O.L.F.O.  approach,  and  the  asyaptotic  behavior  of  the  control 
systea  was  discussed.  There  are  still  sc-re  iaportant  questions  which  have 
not  been  treated  theoretically.  For  exaaple,  the  rate  of  convergence  of  the 
suboptiaal  control  systea  is  in  general  of  great  interest,  but  was  not 
treated  in  detail.  Corputer  studies  vere  carried  out  on  soae  specific 
exaaples  of  third  order  systeas.  The  aain  purpose  for  these  studies  is  to 
provide  us  with  soae  qualitative  ide«s  about  the  rate  of  convergence  of 
the  suboptiaal  control  systea  for  different  types  of  third  order  plants. 

Let  us  consider  a  stochastic  continuous  tine-invariant  linear  systea 
described  by: 


x-(t)  =  A-  (t)  +  bf  uf(t)  +  d_  (t) 
v_(t)  =  c'  +  nf (t) 


x(0)  'v  C(0,  Z  ) 

—  -  —  — XO 

* '  4o> 


(7.1) 


where  Cf(t)  is  a  scalar  driving  white  Gaussian  noise,  qf(t)  is  the 
scalar  observation  white  Gaussian  noise.  The  statistical  laws  of  |-(t) 
and  Tj^(t)  are  assuaed  to  be  known: 


/  2 

if(t)dt  ^  G(0,  /  r  dtj 

(7.2) 

fcl 

nf(t)dt  ^  G(0,  /  q  dt) 

(7.3) 

C1 
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Fron  (7.1),  we  have 


A,t  t  A_ (t  -  t)  t  Ap(t  ~  x) 

x-(t)  =  e  x^(0)  +  /  e  b^uf(->  dx  +  f  r  cL.£f( 

t  Q  —It  '0  — It 


Assume  that  we  cake  observations  only  at  discrete  instants  of  tine 
2A,  3d  d  is  assuned  to  be  snail  such  that  u(t)  =  u(kd),  C(t) 

t  s  [kd,  (k+l)dj : 

A,d  Af(kd)  kd  A- (kd  -*  t) 

xf  (k+l)d  e  fe  xf(0)  +  fQ  e  bp  uf(x)dx 

kd  A.(kd  -  x) 

+  /0  e  ‘  5f(T)dx] 

d  ApO  d  ApO 

+  /  o  e  do  •  bp  Up (kd)  +  /Q  e  do  •  d_  ?f(kd) 


Defining 

Afd  d  ApC 

x(k)  =  x^(kd)  ;  A  =  e  r  ;  b  *  /_  e  do  *  b 


>  «  «■  }  u  -  J  Q  V  UU  —'f_ 

-  AfC 

d  =  fQ  e  do  •  df  ;  5(k)  *  £p(kd)  ;  u(k)  *  uf(kd) 


(7.5)  becones 


x(k+l)  =  A  x(k)  +  Jb  u(k)  +  ci§  (k)  ;  x(0)  Q(0,  £  ) 


Defining 


y(k)  *  yf(kd)  ;  n(k)  *  nf(kd) 


The  observation  sequence  is 


y(k)  *  c.'x(k)  +  n00 


The  statistical  laws  of  C(k),  n(k)  are 


x)dx  (7.4) 

t  =  A, 

=  C(kd), 


(7.5) 

(7.6) 

(7.7) 

(7.8) 

(7.9) 


COO  *  Q(0V  rd) 


(7.10) 
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n(k)  ^  0(0,  qA)  (7.11) 

The  gain  vector  is  assumed  to  be  unknown  but  constant,  therefore  the 
equation  for  the  unknown  gain  is 

b(k+l)  =  b(k)  ;  b(0)  v  CO^  2feo)  (7.12) 

We  can  now  apply  the  results  in  chapter  6  to  equations  (7.6),  (7.9),  (7.10)- 
(7.12). 

A  computer  program  was  designed  which  operates  as  follows: 

(1)  Read  in  A^.,  j>p  _c,  d^,  r,  q,  ,  x^,  b^,  the  final  tine  N  and 

the  different  weightings  W,  h,  F,  and  covariances  ^  • 

(2)  A  subroutine,  which  was  developed  by  Levis  {75],  was  used  to 
convert  the  continuous  version,  (7.1),  to  the  discrete  tine 
sample  data  version  (7.6).  The  covariances  of  £(k),  n(k)  are 
computed  using  (7.10),  (7.11). 

(3)  The  true  value  of  x(k)  was  recorded.  Using  a  noise  generating 
subroutine,  a  sample  value  of  £00  was  obtained.  Assume  that 
^(k-l/k-1),  6(k-l/k-l)  are  recorded.  A  subroutine  for  the 
identification  equations  (6.3,19)-(6.3.23)  was  used  to  obtain  the 
current  estimates  £(k/k),  15(k/k),  and  the  error  covariance 
matrix  j£(k/k)  recursively.  These  values  ware  also  recorded. 

(4)  A  subroutine  based  on  (6.3.32)-(6.3.41)  was  used  to  obtain  the 

*  . 

adaptive  control  u  (k;. 

(5)  The  control  u  (k)  was  applied  to  the  system  (7.6),  using  a 
noise  generating  device  to  obtain  a  sample  value  of  5(k);  then 
by  (7.6),  we  obtained  the  value  jc(k+1). 

(6)  We  advance  k  ->■  k+1  and  repeat  (3)  through  (5)  until  we  get  to 
the  final  time  k  =  N-l. 


1 
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The  program  was  written  in  such  a  way  that  if  we  set  jj(k/k)  =  _b,  and 
=  0,  then  the  procedures  (3)  through  (6)  will  give  us  the  truly  optimal 
stochastic  control  when  _b  is  known.  Using  a  plotting  subroutine  we  can 
plot  out  the  truly  optimal  trajectories  vs.  the  O.L.F.O.  trajectories;  the 
true  _b  vs.  the  estimated  Jb,  and  optimal  feedback  gain  vs.  adaptive  gain 
(it  was  noted  that  the  adaptive  correction  term  will  converge  to  zero  quite 
fast),  under  the  requirement  that  the  same  noise  samples  (£(k),  rj(k))  were 
used  for  both  the  known  and  unknown  _b  cases.  These  plots  provide  us 
with  qualitative  understanding  on  the  rate  of  convergence  of  the  overall 
srboptimal  O.L.F.O.  control  system. 

In  all  the  computer  simulations,  unless  otherwise  mentioned,  we  set 


the  values: 


A  =  0.2  sec,  r  =  0.05,  q  =  0.45,  d^  =  |^2J  ,  Xq  ~  » 

l  “  I3.  «  -  i3.  4.-  ‘  Ly  £'  •  12  0  0) 


(7.13) 


Example  1:  Unstable  System 


It  is  assumed  that 


0  1  °1  r  * 

=  001  ;  b  =  2 

5  -3  -1J  1 1  b-3. 


h  =  1 


Xf(0) 


=  -1  (7.14) 


such  a  system  has  a  transfer  function  (see  Fig.  7.1) 


H,  (s)  -  - 3)(s+  2) - 

1  (3  -  l)(s  +  2s  +  s) 


so  that  it  has  an  unstable  pole  at  s  =  1.  Initially,  we  set 


b^O/O)  =  0 


(7.15) 


(7.16) 


UNSTABLE  SYSTEM 


Fig,  7. lh  POLE  ZERO  PATTERN  FOR  EXAMPLE 


2 :  STABLE  SYSTEM 


X(i> 
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Fig.  7.2  COMPARISON  BETWEEN  THE  OPTIMAL  TRAJECTORY  WHEN  THE  GAIN  IS  KNOWN 

AND  THE  O.L.F.O.  TRAJECTORY  ASSUMING  THE  GAIN  IS  UNKNOWN.  THE  SYSTEM 
BEING  CONTROLLED  IS  UNSTABLE  WITH  SYSTEM  FUNCTION  (S4-3)  (S+2) 

(S-l)  (S^+2S+5) * 

THE  SAMPLE  NOISE  IS  THE  SAME  FOR  BOTH  CASES, 


GRAPH  3 


{  i 


Fig.  7.2  (Continued) 
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Fig.  7.2  (Continued) 


ESTIMATE  OF  THE  UNKNOWN  GAIN  VECTOR.  THE  SYSTEM  BEING 
CONSIDERED  HAS  SYSTEM  FUNCTION  (S+3)  (S+2) 

(S-l)  (S2+2S+5) 


Fig.  7.3  (Continued) 
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fig.  7.3  (Continued) 
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i.e.  we  started  out  with  an  initial  guess  that  the  system  has  no  zeroes. 
The  final  time  is  N  =  40. 

Many  computet  runs  have  been  made  on  the  same  system  with  different 
noise  samples.  The  plots  for  one  particular  sample  experiment,  which 
represents  a  fairly  good  average  behavior,  are  shown  in  Figs.  7. 2-7. 4. 

From  the  experimental  data  (which  are  not  shown  completely),  we  can  obtain 
a  rough  idea  about  the  behavior  of  the  suboptimal  O.L.F.O.  control  system. 

From  the  experiments,  it  was  found  that  in  the  beginning,  the  O.L.F.O. 
adaptive  gain  is  approximately  zero  (Fig.  7.4)  and  the  O.L.F.O.  trajectory 
follows  closely  to  the  input-free  trajectory  (Fig.  7.2).  The  diverging 
phenomenon  is  detected  by  the  identifier;  controls  of  considerably  high 
magnitude  are  then  applied  for  a  few  steps.  This  is  indicated  by  the  fact 
that  there  are  sharp  jumps  in  the  state  trajectories.  Experiments  show 
that  these  jumps  are  not  caused  by  bad  noise  sample  because  the  same 
phenomenon  appears  in  different  sample  runs  at  approximately  the  same  time 
interval.  The  high  magnitude  control  serves  mainly  for  identification 
purposes .  this  is  revealed  by  the  fact  that  at  the  next  time  unit,  the 
estimates  of  ib  closely  agree  with  the  true  b^  (Fig.  7.3).  As  was 
predicted  in  chapter  6.  section  6.3,  the  O.L.F.O.  adaptive  gains  do  con¬ 
verge  to  the  truly  optimum  gains  (Fig.  7.3).  The  correction  term  vs.  time 
is  not  shown  in  the  figure,  but  simulation  results  indicate  that  the 
correction  term  goes  to  zero  very  rapidly  after  the  identification  of  t> 
is  essentially  completed. 

Another  set  of  simulation  experiments  was  carried  out  where  we  kept 
the  same  sample  noise  but  varied  the  weighting  h,  (h  >  0) .  It  was  found 
from  the  experiments  (not  reported  in  here)  that  the  maximum  magnitude  of 
the  overshoot  in  the  O.L.F.O.  trajectories  varied  inversely  with  the  value 


I 
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of  h;  if  h  was  large,  we  have  relatively  "lower"  overshoots;  whereas,  if 
h  was  small,  we  had  relatively  high  overshoots.  Also,  the  experiments  seen 
to  indicate  that  the  convergence  rate  and  the  final  estimation  error  in  _b 
seem  to  depend  on  the  value  of  h  we  chose;  with  large  h,  we  have  relative¬ 
ly  slow  convergence  rate  and  relatively  big  final  estimation  error  in  _b; 
if  h  is  small,  we  have  a  relatively  fast  convergence  rate  and  relatively 
small  final  estimation  error  in  b^. 

In  the  next  set  of  experiments,  we  kept  the  weighting  fixed  (h  =  0.1), 
and  repeated  the  first  set  of  experiments  with  larger  driving  noise  co- 
variance  (r  =  G.45)  while  using  the  same  observation  noise  sample.  The 
experimental  results  (not  reported  in  here)  seem  to  indicate  that  the  in- 
crease  in  driving  noise  covariance  has  little  effect  on  the  convergence 
rate  of  the  O.L.F.O.  control  system. 

It  is  of  interest  to  find  out  whether  the  initial  guess  on  b^  will 
be  sensitive  to  the  resulting  O.L.F.O.  control  system.  We  carried  out  a 
set  of  experiments  where  we  fixed 

"0  1  O' 

Ac  =  0  0  1  (7.17) 

|_5  -3  -lj 


The  transfer  function  is 


h:(s)  = - 5 - 

(s  -  1) (s  +  2s  +  s) 


(7.18) 


The  initial  condition  on  x,-(0)  was  kept  fixed,  and  using  the  same  sample 
noise,  we  varied  our  initial  guess  in  b^.  The  same  runs  seem  to  indicate 
that  though  the  sanole  O.L.F.O.  trajectory  varied  with  different  initial 


esses  in  b,.;  the  convergence  rate  was  quite  insensitive  to  the  guess  in 
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I 

(4  see) 


TIME  UNIT 
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The  true  transfer  function  for  the  system  is  (Fig.  7.1) 


(7.19) 


H 2(s)  » 


(s  +  3)  (s  +  2) 
is  +  l)(s2  +  2s  +  s) 


(7.20) 


The  system  is  stable. 

In  the  first  set  of  experiments,  we  initially  guess 

2 

£,{0/0)  =  1  (7.21) 

b6 

7  /-39  7  /-39 

i .  e .  that  the  zeroes  are  located  at  -  -jr  +  V  —j~  and  -  -j-  -  v  — .  The 
weighting  on  the  control  is  h  =  1.  We  take  the  final  time  N  =  40. 

Sample  runs  for  the  same  system  with  same  initial  guess  (7.21)  were 
cade  and  the  plots  for  one  particular  sample  are  shown  in  Figs.  7. 5-7. 7. 

As  opposed  tc  the  unstable  case,  the  O.  L.  F.  Q.  adaptive  gain  is  some  nonzero 
vector,  and  so  the  value  of  the  O.L.F.O.  control  is  not  zero  at  the 
beginning  (Fig .  7.7).  The  control  is  used  both  for  identification  and 
control  purposes.  The  system  is  stable,  and  since  no  large  magnitude 
control  is  applied,  the  O.L.F.O.  trajectory  decays  down  to  zero  (see 
Fig.  7.5).  This  decaying  phenomenon  is  noticed  by  the  identifier,  and 
thus  the  control  is  kept  near  zero  to  save  energy.  Therefore,  after  a 
certain  time  interval,  when  the  O.L.F.O.  trajectory  goes  near  the  origin, 
the  O.L.F.O.  control  will  remain  zero  for  most  of  the  time.  The  system 
behaves  almost  like  an  input-free  system.  In  fact,  this  is  also  what  the 
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truly  optimum  systen  will  do.  We  note  from  Fig.  7.6  that  the  identification 
process  of  the  unknown  gain  stops  at  about  k  =  20,  which  is  the 
approximate  tine  unit  when  the  O.L.F.O.  state  trajectory  begins  to  stay 
around  zero .  If  we  consider  control  over  an  infinite  interval  (say  using  a 
window-shifting  approach)  we  nay  expect  awfully  slow  convergence  rate  in  the 
estimation  of  b  to  the  true  b,  and  a  slow  convergence  rate  of  O.L.F.O. 
control  systen  to  truly  optimum  control  systen. 

In  the  second  set  of  experiments,  wa  have  the  sane  noise  samples  as 
before  but  starting  'with  the  inif  lal  condition 


xf  (0) 


(7.22) 


The  initial  guess  on  b.  teas 

1 


bf(0/0) 


(7.23) 


i.e.  there  are  no  zeroes.  The  weighting  on  the  control  is  h  =  1,  and  we 
take  the  final  tine  S  =  60.  The  plots  for  one  typical  sample  experiment 
are  shown  in  Figs.  7.8-7.10.  (The  sample  noise  for  the  sample  run  shown 
in  Figs.  7.8-7.10  is  the  same  as  that  showm  in  Figs.  7. 5-7. 7.)  Comparing 
this.  set  of  experiments  with  the  last ,  we  note  that  more  or  less  the  same 
phenomenon  occurred  in  both  sets  of  experiments.  The  final  estimate  in  b> 

*  A 

is  way  off  its  true  value,  in  fact  b^k/k)  and  b7(k/k)  are  opposite  in 
sign  with  those  of  b^  and  b^  respectively;  but  interestingly  enough 
the  adaptive  gains  are  adjusted  accordingly  so  that  the  values  of  the 
O.L.F.O.  control  sequence  and  the  truly  optimal  control  sequence  are  almost 
the  same.  This  set  of  experiments  indicates  yet  slower  convergence  (if 


there  is  any) . 


(4s«cl 


TIME  UNIT 


7.8  COMPARISON  BETWEEN  OPTIMA].  TRAJECTORY  WHEN  THE  GAIN  VECTOR  IS  KNOWN  AND 
THE  O.L.F.O.  TRAJECTORY  ASSUMING  THE  GAIN  VECTOR  IS  UNKNOWN.  THE  SYSTEM 
B^I'T-  CONTROLLED  HAS  SYSTEM  FUNCTION  (S+3)  (S+2)  .  WE  GUESS  INITIALLY 

(S+l)  (S2+2S+3) 

THAT  THERE  ARE  NO  Z  EROES.  THE  NOISE  SAMPLE  IS  THE  SAME  FOR  BOTH  CASES. 
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ESTIMATE  OF  THE  GAIN  VECTOR.  THE  SYSTEM  BEING  CONSIDERED 
HAS  SYSTEM  FUNCTION  (S-F3)  (S-4-2)  THE  INITIAL  GUESS  IS 
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'ote  that  in  both  sets  of  experiments  even  if  the  estinate  of  does 
not  converge  to  the  true  b,  the  truly  optimal  trajectory  and  O.L.F.O. 
trajectory  are  almost  the  same  after  the  transient  period. 

Intuitively,  the  results  are  reasonable.  Since  we  have  not  told  the 
problem  to  identify  ]>,  it  will  not  do  so  unless  the  identification  is 
absolutely  necessary.  The  experimental  results  verified  our  theoretical 
deduction  (see  chapter  6,  section  6.6). 

The  experiments  seem  to  indicate  that  for  stable  system,  the  choice  of 
initial  guess  will  not  greatly  influence  the  O.L.F.O.  trajectory,  but  will 
affect  the  convergence  rate  for  the  estinate  in  the  gain  parameters,  _b. 

Remark:  In  each  set  of  experiments  discussed  above,  the  number  of 
sample  runs  is  not  enough  to  enable  us  to  draw  specific  statistical  con¬ 
clusions;  yet  the  regularity  in  the  sample  runs  enable  us  to  draw  some  crude 
conclusions. 

From  the  experiments,  we  may  draw  the  following  conclusions  regarding 
the  O.L.F.O.  control  system. 

(1)  The  rate  of  convergence  seems  to  be  very  dependent  on  the 
stability  of  the  system.  For  unstable  systems,  the  convergence 
rate  seems  to  be  faster  compared  to  that  for  stable  systems. 

This  verifies  our  theoretical  predictions  made  in  chapter  6, 
section  6.6. 

(2)  It  seems  that  large  controls  will  help  identification  of  the 
unknown  gain  parameters,  and  so  convergence  rate  seems  to  relate 
directly  to  the  magnitude  of  the  control  action.  This  again 
agrees  with  our  intuitive  remark  made  in  chapter  6,  section  6.3. 

(3)  For  unstable  systems,  the  rate  of  convergence  seems  to  be 
fairly  independent  of  the  initial  guess  on  the  unknown  gain, 
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wh-sreas  for  stable  systems,  the  convergence  rate  may  be  quite 
dependent  on  the  initial  guess  on  the  unknown  gain. 

(4)  For  unstable  systems,  the  Q.L.F.O.  trajectory  will  depend  on  the 
the  initial  guess  in  b^,  but  then  for  stable  systems,  the  O.L.F.O. 
trajectory  will  not  vary  drastically  when  we  vary  the  initial 
guess  in  b^ . 

(5)  For  the  unstable  system,  the  O.L.F.O.  trajectory  seems  to  follow 
closely  its  input-free  trajectory  in  the  beginning,  until  the 
diverging  phenomenon  tells  the  identifier  to  send  back  large 
controls  for  identification  purposes.  This  causes  some  overshoots 
in  the  trajectory.  The  magnitude  of  the  maximum  overshoot  seems 
to  relate  inversely  with  the  values  for  the  weighting  constant  h 
on  control.  For  stable  systems,  simultaneous  identification  and 
control  seem  to  be  carried  out  in  the  beginning.  Since  the  system 
is  stable,  with  little  control  energy,  the  state  will  go  to  zero, 
so  after  soma  time  period  when  the  state  is  near  the  origin, 
approximately  zero  contral  is  applied  thus  terminating  the 
identification  of  1). 

(6)  Lastly,  the  author  would  like  to  comment  on  the  computational 
feasibility  of  the  proposed  scheme.  The  above  experiments  were 
simulated  using  an  IBM  360/64/40  system.  It  was  found  that 
the  actual  computation  of  the  O.L.F.O.  control  sequence  can  be 
carried  out  almost  in  real  time  for  N  =  40;  i.e.  in  about  0.2 
seconds,  the  following  tasks  were  accomplished:  One  step  computa¬ 
tion  of  (6.3.19)— (6 .3.32)  (6  vector  difference  equation  and  6  x  6 
matrix  difference  equation),  the  parameter  computations  (6.3.34)- 
(6.3.37),  and  the  computation  of  K(k/k)  (6.3.32),  £(k) (6.3 .39) 
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(one  12  x  12  matrix  difference  equation  and  one  3><3  matrix  difference 
equation,  computed  in  a  time-backward  direction  directly  for  k  <  40  steps, 
k  =  0,  1,  N-l) . 

Further  Experimental  Studies 

The  following  experiments  are  suggested  so  as  to  provide  a  deeper 
understanding  on  this  class  of  problems. 

(1)  Implement  a  window-shif ting  O.L.F.O.  control  sequence  as  was  sug¬ 
gested  in  section  6,7.  This  will  allow  vs  to  consider  control 

over  an  infinite  time  span  for  k  =  0,  i .  To  increase  the 

convergent  rate,  apply  control  sequence 

*  u* (k)  if  u*(k)  >  e 

u  (k)  =  *  (7.24) 

e  if  u  (kj  <  e 

if  ||z°(k/k)i|  and  u*(k)  =  u*(k)  if  ||z£(k/k)||  <  ?. 

The  values  for  6  and  e  are  adjusted  through  experimentation. 

(2)  Design  a  computer  program  which  will  enable  us  to  study  the 
statistical  behavior  of  the  O.L.F.O.  control  system.  For  a 
fixed  assumed  structure  of  the  system  and  the  same  weighting 
constants,  study  the  statistical  behavior  of  the  system  and  the 
average  convergence  rate  of  the  suboptimal  control  system  to  the 
optimal  system.  Vary  the  weighting  constant  h  on  the  control, 
and  investigate,  in  a  statistical  sense,  how  it  affects  the 
average  maximum  overshoot  in  the  trajectory, 

(3)  To  avoid  large  overshoots  in  the  beginning  for  the  unstable 
system,  one  may  wish  to  have  a  large  weighting  factor  h  for 
the  control  energy  in  the  beginning,  and  when  the  true  value  of 


b>  is  exactly  recovered,  we  may  wane  h  =  C.l.  Thus  -e  say 
prefer  h  to  be  tise  varying 

h{k)  =  g(k)  -r  0.1  (7.25) 

«sre  g(k)  is  nsnincreasisg  and  g(k}  0  as  k  ».  Such  an 
od-toc  approach  nay  lead  to  a  veil  behaved  O.L.F.O.  control 
system. 

(4)  Tbs  assumption  that  b  6(b  »  £,  )  is  sade  tor  mathematical  con- 

—  — o  —bo 

venience.  In  actual  practice,  and  say  not  be  available. 

ifith  tbs  results  in  chapter  3,  observability  of  the  pair  (A,  C) 

is  sufficient  to  assure  that  independent  of  the  guess  on  h^, 

asymptotic  convergence  of  the  estimate  of  b  is  obtained.  But 

it  von Id  ba  important  to  find  out  hov  different  assumptions  on 

b  and  I.  vill  effect  the  rate  of  convergence  for  born  stable 
—o  ""’©o 

and  unstable  systems. 

(5)  3y  varying  the  sampling  rate,  one  can  study  the  effect  of  sampling 
period  to  the  behavior  of  the  overall  subop tical  O.L.F.O.  control 


system. 


CHAPTER  VIII 


CONCLUSIONS  AND  SUGGESTIONS  FOR  FURTHER  RESEARCH 

The  observer  theory  for  discrete  and  continuous  time  linear  systems 
have  been  developed  in  parallel.  We  showed  that  one  can  view  an  observer- 
estimator  as  a  learning  device  which  is  used  to  learn  all  recoverable 
uncertainties  while  taking  the  statistical  behavior  of  all  inherent 
disturbances  into  consideration.  The  class  of  observers-estimators  which 
will  do  the  learning  optimally  in  the  mean  square  sense  is  also  derived. 

Such  optimal  classes  of  observers-estimators  can  be  incorporated  in  the 
overall  optimal  control  system,  and  for  this  reason  analytical  studies  on 
the  optimum  class  of  observers-estimators  was  carried  out  in  detail.  It  is 
noted  that  observers  theory  includes  Kalman  filtering  and  deterministic 
exponential  estimation  as  special  cases. 

The  stochastic  control  of  linear  systems  with  known  dynamics  was 
treated  in  detail.  For  this  class  of  problems,  we  have  imperfect  informa¬ 
tion  due  to  the  fact  that  there  are  inherent  noise  disturbance  and  unknown 
initial  condition  of  the  system  being  controlled.  It  was  proved  that  for 
quadratic  criteria  the  optimal  controller  consists  of  a  "learner1’  and  a  set 
of  feedback  gains.  The  learner  is  realized  by  an  optimum  observer-estimator. 
The  result  is  also  known  as  the  Separation  Theorem.  Physically,  the 
operating  function  of  the  optimum  observer-estimator  is  to  learn  the  current 
state  of  the  system.  It  can  be  shown  that  if  the  current  state  of  the 
system  is  asymptotically  recoverable  and  if  the  system  can  be  stabilized 
by  adapting  some  feedback  gain,  then  the  overall  optimal  stochastic  system 
will  have  nice  behavior.  The  approach  taken  in  studying  this  specific, 
class  of  problems  can  be  extended  to  more  general  classes  of  problems 
where  the  cost  criteria  are  other  than  quadratic. 
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In  the  next  level,  we  considered  control  of  linear  discrete  systems 
with  unknown  gain  parameters.  Since  the  truly  optimal  control  sequence 
cannot  be  obtained  because  cf  the  "curse  of  dimensionality."  we  look  for  a 
computationally  feasible  suboptimal  control  sequence.  Prompted  by 
physical  consideration  and  computational  considerations,  we  used  the  open- 
loop  feedback  optimal  approach  to  derive  the  O.L.F.O.  control  sequence.  It 
was  proved  that  the  O.L.F.O.  controller  consists  of  a  learner,  which  we  call 
an  identifier,  and  a  feedback  gain  plus  correction  term.  The  identifier 
is  realized  by  an  optimal  observer-estimator  whose  operating  function  is  to 
learn  the  current  state  and  current  unknown  gain.  Analytical  studies  were 
done  on  the  overall  O.L.F.O.  control  system.  It  was  proved  that  if  the 
initial  state  and  unknown  gain  parameters  are  recoverable,  then  the  overall 
O.L.F.O.  control  system  will  asymptotically  converge  to  the  truly  optima] 
stochastic  control  system.  The  derived  results  seem  to  be  computationally 
feasible.  The  computation  of  the  O.L.F.O.  control  is  done  on-line.  For 
all  time  k  =  0,  1,  ...,  N-l,  we  have  to  compute  a  one-step  2n-vector 
difference  equation  and  a  one-step  2n  >*  2n  matrix  difference  equation 
(identification  equations),  then  a  (N  -  k)-steps  n-vector  difference  equa¬ 
tion  and  a  (N  -  k)-steps  n  *  n  matrix  difference  equation  (parameters 
computation),  and  finally  a  (N  -  k)-steps  (n+l)n  x  (n+l)n  matrix  difference 
equation  (computation  of  k  (k/k) (see  Fig.  6.2  and  Fig.  6.3).  The  vectors 

and  matrices  being  stored  as  time  unit  advances  are  jc(k/k) ,  _b(k/k),  and 

2 

^(k/k)  which  require  a  total  of  (2n  +  3n)  memory  locations.  (Note  that 
Z^(k/k)  is  symmetric  and  this  cuts  down  the  storage  memory  requirements.) 

Using  the  theoretical  results  derived,  a  computer  program  is  developed 
to  study  the  control  of  a  variety  of  third  order  systems  with  known  poles 
but  unknown  zeroes.  Sample  runs  were  made  mainly  to  study  the  convergence 
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rate  of  the  O.L.F.O.  control  system  to  the  truly  optimal  system.  The 
experimental  results  seem  lo  indicate  that  the  rate  of  convergence  depends 
on  the  structure  of  the  system:  stable  plants  appear  to  have  slow  con¬ 
vergence,  whereas  unstable  plants  vrill  result  i.i  fast  convergence.  For 
stable  system,  the  convergence  rate  depends  highly  on  the  initial  guess  of 
the  unknown  zeroes  locations;  but  for  unstable  stable,  it  appears  that  the 
rate  of  convergence  is  quite  insensitive  to  the  initial  guess  of  the 
unknown  zeroes  locations.  More  experiments  must  be  performed  so  as  to 
obtain  a  deeper  understanding  on  this  class  of  problems  and  obtain 
engineering  rules-of-thumb. 

Directions  of  further  research  which  are  related  directly  to  this 
work  are  suggested  near  the  end  of  each  chapter  when  appropriate.  In  the 
following,  a  list  of  topics  is  given,  which  the  author  thinks  is  a  continua¬ 
tion  of  this  present  work.  Some  possible  approaches  to  these  different 
problems  are  suggested  and  the  applicability  of  the  results  obtained  in 
this  thesis  to  these  different  problems  is  discussed. 

(A)  Stochastic  Control  of  Continuous-Time  Linear  Systems  With  Unknown  Gains 

We  consider  a  continuous  analog  of  (6.2.1) 

x(t)  *  A(t)x(t)  +  b(t)u(t)  +  £(t)  ;  x(t  )  ^  G(x  ,  I  ) 

—  —  —  —  —  —  oT  — c  — xo 

S  :  (6.8.1) 

2.(0  =  C(t)x(t)  +  Ji(t) 

the  gain  vector  _b(t)  is  unknown  but  satisfies  the  stochastic  differential 
equation: 

b(t)  =  G(t)b(t)  +  x(t)  :  b(tQ)  -v  Q(b^,  Ebo)  (6.8.2) 

The  noises,  £(t),  _n(t),  and  %(t)  are  assumed  to  be  white  Gaussian  with 
known  statistical  law.  The  performance  measure  is 


.,'"(u)  =  E{x'(T)  F  x(T)  +  /  [x'(t)  W(t)  x(t)  +  u“  (t)h(t)]  dt|  (6.8.3) 

t 

o 

The  control  problem  is  to  find  u(t),  t  c  [tQ,  T) ,  such  that  (6.8.3)  is 
minimized  subject  to  dynamic  constraints,  (6.8.1)  and  (6.8.2).  Instead  of 
first  taking  a  sample  data  version  of  the  problem  and  then  applying  the 
derived  results  in  chapter  6  (see  chapter  7),  ue  can  apply  the  O.L.F.O. 
approach  to  the  continuous  time  system  directly.  One  would  then  obtain  a 
continuous  time  identifier  which  estimates  the  current  state  and  current 
gain  in  continuous  time.  The  results  in  chapter  4  can  be  applied.  As 
analogous  to  the  discrete  time  version,  we  would  then  formulate  a  deterministic 
(continuous-time)  open-loop  control  problem.  One  may  expect  the  overall 
O.L.F.O.  control  system  in  the  cor.tinuous-time  case  will  be  similar  in 
structure  to  that  in  the  discrete  version.  The  main  difficulty  lies  in  the 
capability  of  computing  the  O.L.F.O.  adaptive  gain  and  the  correction  term 
in  continuous  time.  Some  modifications  can  be  made  which  take  computation 
capability  into  account.  One  approach  may  be  that  we  resolve  the  open-loop 
problem  only  in  discrete  time,  t  =  0,  A,  2A,  3A,  ...,  even  though  we  have 
continuous  time  observation. 

(B)  Control  With  Unknown  Dynamics 

Consider  the  problem  of  controlling  an  unknown  system  §,  (6.2.1), 
where  the  matrix  A(k) ,  k  ~  0,  1,  2,  ...,  is  unknown  but  satisfies  some 
linear  difference  equation.  The  statistical  laws  of  the  noise  are  assumed 
known.  Our  objective  is  to  control  the  system  §  using  the  quadratic 
criteria.  Formally,  the  truly  optimal  control  can  be  obtained  if  we  can 
solve  Bellman's  equation.  Unfortunately,  this  is  impossible  with  the 
present  stage  of  development  of  computer  technology.  Therefore,  one  can 
look  for  suboptimal  but  computationally  feasible  solutions  to  the  problem. 
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It  would  be  desirable  if  we  can  have  analytical  studies  on  the  derived 
suboptimal  control  system.  Different  approaches  guided  by  engineering 
intuition  are  possible.  An  approach,  which  is  a  combination  of  maximum 
likelihood  and  O.L.F.O.  is  suggested  where  an  analytic  study  of  the  be¬ 
havior  of  the  overall  suboptimal  system  may  ba  possible. 

Consider  the  augmented  system  g  given  by  (6.5.24).  Let  U(0,  k-1) 
be  applied  and  Y^^  y  ^  (0,k)  is  observed.  The  most  probable  estimate 
^maximum  likelihood  estimate^  of  |  A(i)J  *  \  which  is  denoted  by  |A^(i)|^  ", 
is  obtained  by  picking  |A(i)|*  ^  to  maximize  the  conditional  probability 
density  p^|A(i)J^  k-l)^’"^)  su^3ect  to  2  certain  difference  equa¬ 

tion  describing  the  evoluation  of  A(i) ,  i  =  0,  1,  . ..,  k-1.  Extrapolate 

the  estimate  of  |A(i)J  and  the  estimates  are  denoted  by  |A.°(i)lm 

'i=k  fi=k* 

Assume  that  |A^(i)}”_0  is  clie  true  — ^  =  •••»  and  apply  the 

results  of  chapter  6.  The  whole  procedure  is  repeated  at  every  step. 


k—  0,  1,  ....  . 

Theoretically,  this  approach  has  some  advantageous  features.  Using 

Wald's  Theorem  [68],  one  will  obtain  asymptotic  consistent  (with  probability 

1)  estimate  of  {  A(i)|  ;  one  can  then  apply  the  results  of  section  6.6  to 

i=0 

obtain  overall  asymptotic  optimal  control  system. 

The  difficulty  lies  in  the  real  time  computation  of  {A^d)}” _  , 

k  =  0,  1 . using  a  computer.  For  references  in  maximum  likelihood 

estimation,  see  Kashyap  [67],  Wald  [68],  Rauch,  Tung,  and  Striebel  [44]; 

■'or  evaluation  of  likelihood  functions  of  a  Gaussian  process,  see  also 
Schweppe  [69] . 

(C)  Control  With  Unknown  Gain  and  Imperfectly  Known  Disturbance 

Assume  that  the  matrices  A(k) ,  k  =  0,  1,  ...,  are  known,  the  gain 
vectors  jj(k) ,  k  =  0,  1,  are  assumed  to  be  unknown  but  described  bv 


(6.2.2),  (6.2.4),  and  (6.2.7).  The  vectors  _n(k),  C^(k),  Y  (k) ,  k  =  0,  1,  ... 
are  independent  Gaussian  vectors  with  unknown  means  and/or  covariances.  It 
is  necessary  for  us  to  recover  the  true  means  and  covariances  of  the  noise 
vectors.  A  combination  of  maximum  likelihood  and  O.L.F.O.  approach  can  be 
applied  to  such  class  of  problems. 

For  references  which  are  related  to  this  class  of  problems,  see  Saga 
and  Husa  170],  Taran  171],  Kashyap  [67]. 

With  some  thorough  understanding  in  the  problems  (A)  and  (B) ,  we  can 
then  start  to  investigate  the  problem  of  controlling  a  system  where  A(k), 
b (k) ,  k  =  0,  1,  ....  are  unknown  but  satisfy  some  difference  equations,  and 
the  noise  vectors  are  independent  Gaussian  vectors  with  unknown  means  and 


variances . 


APPENDIX  A 


ON  THE  FSEUDO-INVERSE  OF  A  MATRIX 


Let 

A  be  an  n  * 

A  is 

denoted  by  A1 

(1) 

A  Ax  =  x  ; 

(2) 

fj'z  =  0  ; 

(3) 

+  z)  =  A 

With 

this  definite 

(A) 

/.if  J  . 

(A  )  ■  A 

(B) 

Al‘  A  A"  =  A^ 

(C) 

A  /  A  =  A 

(E) 

Let  A  be  an 

V  x  e  Ra(0 

V  2  E  N(A>) 

#  .  J 


A1'  =  (A'  A)"1  A  ' 


-inverse 


(A.l) 
(A. 2) 
(A.  3) 

(A. 4) 
(A.  5) 
(A. 6) 

(A. 7) 


For  the  proofs  of  (A.4)-(A.7),  see  Zadah  and  Desoer  (48],  i.evine  [23];  for 
a  different  approach  to  generalised  inverse  of  a  matrix,  see  Penrose  [72], 


APPENDIX  B 


KEINER-liOPF  r  '  -JATION 

Let  F(k)  =  F(y(i);  i  =  0,  1,  . ..,  k),  we  have  F(i)C  F(i+1), 

1  =  0,  1,  k-1,  and  so  £(i)  is  F(k) -measurable,  for  i  =  0,  1,  k. 

Using  lemma  2.2.6,  and  lemma  2.2.7,  ’.e  have 

Ejx(k)  Z'Ci)}  =  E  |  E|x(k)  £-(i)/F(k)}j  =  E]x(k/k)  £'(i)}  (B.l) 

i  ■■  1  y  »  «  •  j  k 

By  assumption,  wGO  satisfies  (3.3.18),  thus 

Ej(w(k)  -  x(k/k))x"(i)[  =0  i  *  0,  1,  ...,  k  ;  k  =  0,  1,  ....  (B.2) 

Since  both  w(k),  x(k/fc)  are  linear  functioals  of  £(0) ,  ...,  £(k) ,  (B.2) 
also  implies 

E{(w(k)  -  x(k/k))(w(k)  -  x(k/k))"j  =  0  k  =  0,  i .  (B.3) 

Thus  w(k)  =  x(k/k)  a.s. 

The  proof  of  Weiner-Hopf  equation  for  the  continuous  case  is  the 

same  with  slight  modification,  the  induced  c-algebra  F(k)  is  replaced 

by  Ffc  =  Fj^Cx),  x  e  i'to,  t),  i2(i),  x  e  [tQ,  t]|.  And  so  if 

w(t);  t  ->  t  is  a  random  process  such  that  for  t  >  t  ,  w(t)  is  a 
—  —  o  —  o 

linear  functional  of  ^(x),  x  e  [tQ,  t),  and  £2(x)  >  t  t  [tQ,  t ] ;  and 
w(t)  satisfies 

E|w(t)  X((t)|  =|E  x(t)  y£(x)} 

E|w(t)  £2(x)|  ={E  3c(t)  ^'(t)} 

then  w(t)  =  x(t/t)  a.s.,  t  >_  tQ.  (B.4),  (B.5)  imply  the  projection 
equations  (4.3.44). 


T  G  It  * 

t)  ;  t  >  t 

(B.4) 

0 

—  0 

t  e  [tQ, 

t]  :  t  >  t 
—  o 

(B.5) 
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APPENDIX  C 


EQUATION  FOR  ERROR  PROCESS  (CONTINUOUS  TIME  CASE) 

Let  x(t)  be  a  random  process  given  by  (4.3.1),  and  w(t)  be  a 
random  process  satisfying  (4.3.19),  (4.3.20),  and  (4.3.16).  Define 
_e(t)  =  w(t)  -x(t).  Differentiating  _e (t)  and  using  (4.3.19),  (4.3.1), 
and  (4.3.12): 

e(t)  =  P(t)  z(t)  +  P(t)  T(t)  A(t)  w(t)  -  P(t)  T(t)  L  (t)  C^t)  v(t) 

+  P(t)  T(t)  P(t)  z(t)  +  P(t)  T(t)  V2(t)  C2(t)  x(t) 

+  P(t)  T(t)  I^Ct)  Cx(t)  x(t)  +  P(t)  T(t)  L  (t)  a(t) 

+  P(t)  T(t)  B(t)  u(t)  +  V2(t)  C2/t)  x(t)  +  V2(t)  C2(t)  x(t) 

•b  V2(t)  C,(t)  A(t)  x(t)  +  V2(t)  C2(t)  B(t)  u(t)  +  V2(t)  C2(t)  C_(t) 
-  A(t)  x(t)  -  B(t;  u(t)  -  Kit) 

=  (p(t)  +  P(t)  T(t)  P(t)j  z(t)  -  (^(t)  T(t)  +  P(t)  T(t)  P(t)  T(t)) 

•  x(t)  +  P(t)  T(t)  A(t)  e(t)  -  F(t)  T(t)  L  (t)  C^t)  e(t) 

+  P(t)  T(t)  Lx(t)  n(t)  -  ?(t)  Tit)  Kit)  (C.l) 

Since  P_(t) ,  T(t),  V2(t),  £2(t)  also  satisfy  (4.3.16),  we  have 

T(t)  P(t)  +  T(t)  £(t)  =  0  ;  C2(t)  P(t)  +  C2(t)  P(t)  -  0  (C.2) 

and  so  we  have 

£(t)  +  P(t)  T(t)  P(t)  -  (i^  -  P(t)  T(t))  P(t)  =  V2(t)  C2(t)  P(t) 

=  -  V2(t)  C2(t)  P(t)  (C.  3) 

P(t)  T(t)  +  P(t)  T(t)  P(t)  T(t)  =  -  V2(t)  C2(t)  P(t)  T(t) 

=  -  V2(t)  C2(t)  +  V2(t)  C2(t)  V2(t)  C2(t)  (C.4) 
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Subs  ti  tuting  (C.3)  and  (C.4)  into  (C.l)  and  noting  u(t)  =  P^(t)  £(t) 

+  X2(t)  £2(t),  we  have 

e(t)  =  (A(t)  ~  V2(t)  C2(t)  -  P(t)  T(t)  L,(t)  C^t))  e(t) 

+  (V,(t)  C.(t)  -  I  )  KM  +  P(t)  T(t)  L.(t)  n(t)  (C.O 

— 2  —2  — n  —  —  — i  — 

The  initial  error  is 

e(t  )  =  P(t  )  z(t  )  +  V_(t  )  C  (t  )(x(t  )  -  x(t  )) 

—  O  —  o  —  o  2  o  2.  O  '  0  o 

=  P(t  )  T(t  )(x  -  x(t  ))  =  (!-  V,(t  )  C_(t_))(x  -  x(t  ))  (C.6) 

—  0  0  — o  —  o  — n  2  o  2  0  o  o 


BIBLIOGRAPHY 


1.  Doob,  J.  L.,  Stochastic  Processes,  Wiley  and  Sons,  Inc.,  New  York, 

1953. 

2.  Lo&ve,  M.  Probability  Theory,  Van  Nostrand,  New  York,  3rd  edition, 

1963.  .  . 

3.  Davenport,  W.  B.,  Jr.,  and  Root,  W.  L.,  Random  Signals  and  Noise, 
McGraw-Hill  Book  Co.,  Inc.,  1958. 

4.  Wonham,  W.  M. ,  Lecture  Notes  on  Stochastic  Control,  Lecture  Notes 
67-2,  Center  for  Dynamical  Systems,  Division  of  Applied  Mathematics, 
Brown  University,  February  1967. 

5.  Tse,  E.,  "Generalized  Random  Processes,"  Report  ESL-R-366,  M.I.T., 
October  1968. 

6.  Tricomi,  F.  G.,  Integral  Equations,  New  York,  Interscience  Publishers, 
1957. 

7.  Dyr.kin,  E.  B.  .Markov  Process,  Academic  Press,  New  York,  1965. 

8.  Skorokhod,  A.  V.,  Studies  in  the  Theory  of  Random  Process,  Addison- 
Welaey,  New  York,  1965. 

9.  Tse,  E.,  "Application  cf  Pontryagin's  Minimum  Principles  to  Filtering 
Problems,"  M.S.  Thesis,  E.E.,  M.I.T.,  June  1967. 

10.  Athans,  M. ,  and  Tse,  E.,  "A  Direct  Derivation  of  the  Optimal  Linear 
Filter  Using  the  Maximum  Principle,"  IEEE  Trans,  on  Automatic  Control, 
December  1967. 

11.  Pontryagin,  L.  S.s  Boltyanskii,  V.  G.,  Gamkrelidze,  R.  V.,  and 
Mishchenko,  E.  F.,  The  Mathematical  Theory  to  Optimal  Processes, 

New  York,  Interscience,  1962. 

12.  Athans,  M»,  "The  Matrix  Minimum  Principle,"  Information  and  Control, 
Vol.  11,  December  1967. 

13.  Halmas,  P.,  and  Savage,  L.  J.,  "Application  of  the  Radon-Nikodym 
Theorem  of  Sufficient  Statistics,"  Ann.  Math.  Stat,  20,  1949. 

14.  Meyer,  P.  A.,  Probability  and  Potentials,  Blaisdell  Publishing  Company, 
1966. 

15.  Cramer,  H.,  Mathematical  Methods  of  Statistics,  Princeton  University 
Press,  Princeton,  New  Jersey,  1946. 


-281- 


-2S2- 


16.  Mckesn.  3.  ?.,  Stochastic  Integral,  Acadaaic  Press,  1969. 

17.  I  to,  s. ,  ”03  Stochastic  Differential  Equations,  **  Hem.  Amer.  Math. 

Soc.,  4,  1931. 

13.  Gelfamd,  I.  H..  and  Vilenkin,  X.  Y . ,  Generalized  Functions,  Vol.  4, 
Academic  Press,  1964. 

19.  Strantcoovicb,  X-  L-,  "A  Hew  Representation  for  Stochastic  Integrals 
and  Equations,"  SIAM  J.  Control  4(2),  1966. 

20.  Vcns*  S-»  ztd  Zakai,  S.,  ”0a  the  Relation  Letveen  Ordinary  and 
Stochastic  Differential  Equations,"  Int.  J.  Engrg.  Sci.,  3,  1965. 

21.  Clark,  J.  M.  C. ,  "The  Representation  of  Soa-Linear  Stochastic  Systems 
•«ith  Applications  to  Filtering,"  Ph.D.  Thesis,  Electrical  Engineering 
Department,  Imperial  College,  London,  1966- 

22-  Stasaa.  K.  IS.,  "Random  Differencial  Equations  in  Control  Theory."  to 
appear:  A.  1.  3harecfca-3eid,  Ed.,  "Ptababilistic  Jfethods  in  Applied 
Mathematics,"  Vol-  11,  Academic  Press,  1969. 

23-  Levine,  5f.  S. ,  "Optimal  Output-Feedback  Controllers  for  Linear 
Systems,”  Ph.D.  Thesis,  S.I.T.,  January  1969. 

24.  Sens,  A.,  "Optimal  and  Subcptimal  Feedback  Control  for  Deterministic 
Systems,"  Ph.D.  thesis,  E.E.  Department,  M.i.T. ,  February  1969. 

25.  Del loan,  R. ,  Dynamic  Programming,  Princeton  University  Press,  Princeton, 
Heir  Jersey,  1957. 

26.  Dreyfus,  S.  E.,  Dynamic  Programming  and  the  Calculus  of  Variations, 
Mathematics  in  Science  ana  Engineering,  Academic  Press,  1965. 

27.  Vonham,  ».  M. ,  "Stochastic  Problems  in  Optimal  Control, "  RIAS  Tech. 

3pt.  63-14.  May  1963- 

28.  Deyst,  J.  J.,  Jr.,  and  Price,  C.  F. ,  ‘Conditions  for  Asymptotic 
Stability  of  the  Discrete,  Minimum-Variance ,  Linear  Estimator," 

IEEE  Trans-  on  Automatic  Control,  Vol.  AC-13,  Ho.  6,  December  1968. 

29-  Lorenson,  h.  W.,  "On  the  Error  Behavior  in  Linear  Minimum  Variance 
Estimation  Problems,"  IEEE  Trans,  on  Automatic  Control,  Vol.  AC-12, 

Ho-  5,  October  1967. 

30.  Aoki,  M.,  Optimization  of  Stochastic  Systems,  Mathenatics  in  Science 
and  Engineering,  Vol.  32,  Academic  Press,  1967. 

31.  Kleinman,  D.  i— "On  the  Linear  Regulator  Problem  and  the  Matrix 
Riccati  Equation,”  ESL  Report  271,  M.I.T.,  1966. 


-283- 


32.  Wonham,  W.  M. ,  "On  a  Matrix  Riccati  Equation  of  Stochastic  Control," 
SIAM  J.  on  Control,  Vol.  6,  No.  4,  November  3968. 

33.  Holtzman,  J.  M. ,  and  Halkin,  K.,  "Directional  Convexity  and  Maximum 
Principle  for  Discrete  System,"  J.  SIAM  Control,  4,  1966. 

34.  Bellman,  R. ,  Introduction  to  Matrix  Analysis,  McGraw  Hill  Book  Co., 

Inc.,  New  York,  1960. 

35.  Luenberger,  D.  G.,  "Observers  for  Multivariable  Systems,"  IEEE  Trans. 
on  Automatic  Control,  Vol.  AC-11,  April  1966. 

36.  Luenberger,  D.  G.,  "Observing  the  State  of  a  Linear  System,"  IEEE  Trans, 
on  Military  Electronics,  Vol.  MIL-8,  April  1964. 

37.  Aoki,  M. ,  and  Huddle,  J.  R. ,  "Estimation  of  the  State  Vector  of  a 

Linear  Stochastic  System  with  Constrained  Estimator,"  IEEE  Trans,  on 
Automatic  Control,  AC-12,  August  1967.  " 

38.  Tse,  E. ,  and  Athans,  M. ,  "Optimal  Minimal-Order  Observers-Filters  for 
Discrete  Linear  Time-Varying  Systems,"  Submitted  to  IEEE  Trans,  on 
Automatic  Control. 

39.  Kalman,  R.  E.,  "A  New  Approach  to  Linear  Filtering  and  Prediction 
Problems,"  Trans.  ASME,  J.  Basic  Engrg.,  Ser.  D,  Vol.  82,  March  1960. 

40.  Kalman,  R.  E.,  "Fundamental  Study  of  Adaptive  Control  Systems,"  Tech. 
Rept.  ASK-TR-61,  Vol.  1,  App.  I. 

41.  Kalman,  R.  E.,  "On  the  General  Theory  of  Control  Systems,"  Proc. 

First  International  Congress  on  Automatic  Control,  Moscow,  1960. 

42.  Athans,  M. ,  Wishner,  R.  F.,  and  Bertolini,  A.,  "Suboptimal  State 
Estimation  for  Continuous-Time  Nonlinear  Systems  from  Discrete  Noisy 
Measurements,"  IEEE  Trans,  on  Automatic  Control,  Vol.  AC-13,  October 
19  t'i. 

43.  Bryson,  A.,  Jr.,  and  Ho,  Y.  C.  Applied  Optimal  Control,  Blaisdell 
Publishing  Company,  1969. 

44.  Rauch,  H.,  Tung,  G.,  and  Streibel,  "On  the  Maximum  Likelihocd  Estimates 
for  Linear  Dynamic  Systems,"  AIAA  J.,  Vol.  3,  1965. 

45.  Freeman,,  H.,  Discrete-Time  Systems,  John  Wiley  and  Sons,  Inc.,  1965. 

46.  Henrikson,  L.  J.,  "Sequentially  Correlated  Measurement  Noise  with 
Application  to  Inertial  Navigation,"  Spacecraft  and  Rockets,  Vol.  5, 
1968. 


w  vi  ‘  v v  *»■;  W-STfWAW  Wiin  i 1  ^  iu.  •«. -."Tr'V^f.VviV  p^'IpV^ht ^'7-T V’  v.s  JLYi 4u,!*w?W-o'lu^ V,W  ^ J<u  W*1  - 'WMWV* M.-Wl Ml™ WW # 


-284- 


47.  Van  Trees,  H.  L. ,  Detection,  Estimation,  and  Modulation  Theory,  Part  I, 
John  Wiley  and  Sons,  Inc.,  New  York,  1968. 

48.  Zadeh,  L.  A.,  and  Desoer,  C.  A.,  "Linear  System  Theory,"  Me G raw  Hill 
Book  Company,  New  York,  1963. 

49.  Johnson,  G.  W.,  "A  Deterministic  Theory  of  Estimation  and  Control," 
JACC,  1969. 

50.  Kalman,  R.  E.,  and  Bucy,  R.  S.,  "New  Results  in  Linear  Filtering  and 
Prediction  Theory,"  Trans.  ASMS,  J.  Basic  Engrg.,  Vol.  33,  1961. 

51.  Weiner,  N.,  Extrapolation,  Interpolation  and  Smoothing  of  Stationary 
Time  Series,  John  Wiley  and  Sons,  New  York,  1949. 

52.  Bucy,  R.  S.,  "Optimal  Filtering  for  Correlated  Noise,"  J.  of  Math. 
Analysis  and  Application,  Vol.  20,  No.  1,  October  1967. 

53.  Mehra,  R.  K. ,  and  Bryson,  A.  E.,  Jr.,  "Linear  Smoothing  Using 
Measurements  Containing  Correlated  Noise  with  an  Application  to 
Interial  Navigation,"  IEEE  Trans,  on  Automatic  Control,  AC-13, 

October  1968. 

54.  Root,  W.  L.,  "Singular  Gaussian  Measures  in  Detection  Theory,"  Time 
Series  Analysis,  M.  Rosenblatt,  Ed.,  New  York,  J.  W.  Wiley,  1963. 

55.  Geesey,  R.,  and  Kailath,  T.,  "An  Innovation  Approach  tc  Least-Squares 
Estimation — Part  III:  Estimation  in  Colored  Noise,"  to  be  published. 

56.  Joseph,  P.  D. ,  and  Tou,  J.  T.,  "On  Linear  Control  Theory,"  AIEE  Trans. 
(Appi.  and  Industry),  September  1961. 

57.  Athans,  M. ,  and  Falb,  P.,  Optimal  Control,  McGraw  Hill,  New  York,  1966. 

58.  Gunckel,  T.  L.,  and  Franklin,  G.  F.,  "A  General  Solution  for  Linear, 
Sampled-Data  Control,"  Trans.  ASME,  85D,  June  1963. 

59.  Streibel,  C.,  "Sufficient  Statistics  in  the  Optimal  Control  of 
Stochastic  Systems,"  J.  of  Math.  Analysis  and  Appl.,  12,  1965. 

60.  Farison,  J.  3.,  Graham,  R.  E.,  and  Shelton,  R.  C.,  "Identification 
and  Control  of  Linear  Discrete  Systems,"  IEEE  Trans,  on  Automatic 
Control,  AC-12,  August  1967. 

61.  Murphy,  W.  J.,  "Optimal  Stochastic  Control  of  Discrete  Linear  Systems 
with  Unknown  Gain,"  IEEE  Trans,  on  Automatic  Control,  AC-13,  August 
1968. 


-285- 


i 


62.  Goman,  D.,  and  Zaboroszky,  J.,  "Stochastic  Optimal  Control  of 
Continuous  Time  Systems  with  Unknown  Gair.,"  IEEE  Trans-  on  Automatic 
Control,  AC-13,  December  1968. 

63.  Bar-Shalom,  Y.,  and  Sivan,  R.,  "On  the  Optimal  Control  of  Discrete- 
Time  Linear  Systems  with  Random  Parameters,"  IEEE  Trans,  on  Automatic 
Control,  AC-14,  February  1969. 


64  Florentin,  J. J. ,  "Optimal,  Probing,  Adaptive  Control  of  a  Simple 
Raysian  System,"  J.  Electronics  and  Control,  Vol.  II,  1962. 

65.  Witsenhausen,  H.  S.,  "Minima::  C.ntrols  of  Uncertain  Systems,"  Ph.D. 
Thesis,  Department  of  Electrical  Engineering,  M.I.T. ,  May  1966. 

66.  Schweppe,  F.  C.,  Notes  for  the  course  6.606  offered  at  M.I.T. ,  1569. 

67.  Kashyap,  R.  L.,  "Maximum  Likelihood  Identification  of  Stochastic 
Linear  Systems,"  to  be  published. 

68.  Wald,  A.,  "Note  on  the  Consistency  of  the  Maximum  Likelihood  Estimate," 
Ann.  Math.  Stat.,  Vol.  20,  No.  4,  December  1949. 

69.  Schweppe,  F.  C.,  "Evaluation  of  Likelihood  Functions  for  Gaussian 
Signals,"  IEEE  Trans,  on  Information  Theory,  IT-11,  January  1965. 

70.  Saga,  A.  P-,  and  Husa,  G.  W. ,  "Adaptive  Filtering  with  Unknown  Prior 
Statistics,"  JACC,  1969. 

71.  Taran,  T.  J.,  "Stochastic  Optimal  Control  with  Noisy  Observation  having 
Imperfectly  Known  Disturbances,"  JACC,  1969. 

72.  Penrose,  R. ,  "A  Generalized  Inverse  for  Matrices,"  Proc.  Cambridge 
Phil.  Soc.,  51,  1955. 

73.  Fel'dbaum,  A.  A.,  Optimal  Control  Systems ,  Mathematics  in  Science  and 
Engineering,  Vol.  22,  Academic  Press,  1965. 

74.  Kushner,  H.  J.,  Stochastic  Stability  and  Control,  Mathematics  in 
Science  and  Engineering. 

75.  Levis,  A.,  "On  the  Optimal  Sampled-Data  Control  of  Linear  Processes,” 
Ph.D.  Thesis,  M.I.T.,  June  1968. 


i 


I 

i 


i 


& 

V  - 


S'  <  *n!\  •  I.S'.  .l:t  . 


cxument 


Col  "KC*-  D*\Ta  - 


ti  &  D 


>rfi  •  *n  -.if 


•  •V* 


»  o»«  *c  :w/v  «;r.  r^^u*  nur**.)/ « 

Massachusetts  Institute  of  Technology 
Department  of  Eloccrical  Engineering 
Cambridge,  Massachusetts  02139 


UhCLA.ss  Jk'  1  ,:r> 


5 

■  -r 


4.  DESCRIPTIVE  NOTES  (Type  ot  repost  and  Inclusive  dale*) 

Scientific  Interim 

5.  AUlHOmS)  (Firtt  nam*,  middle  initial .  im*t  name} 


Edison  Tack-Shuen  Tse 


c.  RF.PORT  OA7C 

January  1970 

7 a.  TOTAL  NO  RACLS  j  ft*.  ’.D  C*  Rf.^S 

285  i  75 

il.  CONTRACT  OR  CRAlSiTKO- 

AFOSR-69-1724 

6.  PROJECT  NO 

9749-01 

c  611021? 

631304 

9*.  ORIGINATOR’S  REPORT  \UM6(.R<S) 

ESL-R-412 

9b,  OTNE«  REPORT  nOiSI  (Any  othet  numbers  ihol  rr.oy  bo  usalptxed 
this  repot: ) 

AFOSR  70 -0474TR _ 

10  OISTAIOUTION  SIATeuCKI 


1.  This  document  has  been  approved  for  public 
release  and  sale;  its  distribution  is  unlimited 


ii.  supplementary  notes 

12  SPONSORING  military  *C7iy  tv 

Air  Force  Office  of  Scientific  Research  (SR 

TECH,  OTHER 

1400  Wilson  Boulevard 

Arlington,  Virginia  22209 

id  adjt^act 


The  control  of  linear  systems  with  incomplete  information  is  considered 
where  the  unknown  disturbances  and/or  random  parameters  are  assumed  to  satisfy 
some  statistical  laws. 

The  observer  theory  for  linear  systems  is  developed  which  general i zes 
the  concepts  due  to  Kalman  and  Luenberger  pertaining  to  the  design  of  linear 
systems  which  estimate  the  state  of  a  linear  plant  on  the  basis  of  both  noise- 
free  and  noisy  measurements  of  the  output  variables.  The  Separation  Theorem  for 
linear  system  is  then  extended  for  suen  observers-estimators. 

Tiie  problem  of  controlling  a  linear  system  u 1 th  unknown  gain  is  then 
considered.  An  open-loop-feedback-optimal  control  algorithm  is  developed  wnich 
seems  to  be  computationally  feasible.  Existence  of  such  subopt imal  control  sciieme 
is  proved  under  the  assumption  that  the  unvertaintles  in  the  unknown  gain  will 
not  grow  in  time.  Covergence  of  such  suboptimal  control  system  to  the  truly 
optimal  control  system  is  considered.  A  computer  program  is  developed  to  study 
the  control  of  a  variety  third  order  systems  with  known  poles  but  unknown  zeroes. 
The  experimental  results  serve  to  provide  us  with  some  more  insights  into  the 
structure  and  behavior  of  the  open-loop-feedback-ootimal  control  systems. 
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